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Why  GAO  Did  This  Study 


What  GAO  Found 


In  2001,  the  Department  of  Defense 
(DOD)  identified  performance 
based  logistics  (PBL)  as  the 
preferred  weapon  system  support 
strategy.  Within  DOD,  PBL  is  the 
purchase  of  performance 
outcomes,  such  as  system 
availability,  rather  than  the 
purchase  of  individual  elements  of 
logistics  support — such  as  parts, 
repairs,  and  engineering  support. 
Although  PBL  initially  arose  from 
efforts  to  reduce  support  costs, 
questions  have  arisen  about 
whether  PBL  has  reduced  support 
costs  as  originally  intended.  GAO 
was  asked  to  evaluate  the  extent  to 
which  DOD  has  used  business  case 
analyses  to  guide  decisions  related 
to  PBL  arrangements  and  the 
impact  PBL  arrangements  have  had 
on  weapon  system  support  costs. 

In  conducting  the  review,  GAO 
analyzed  the  implementation  of 
PBL  arrangements  for  29  weapon 
system  programs.  GAO  also  looked 
at  the  use  and  characteristics  of 
performance-based  contracting  in 
the  United  Kingdom’s  Ministry  of 
Defence. 


What  GAO  Recommends 


GAO  makes  five  recommendations, 
including  that  DOD  require  the 
development  of  business  case 
analyses,  clearly  define  specific 
criteria  for  their  development,  and 
require  that  the  services  improve 
their  internal  controls  to  ensure 
that  the  analyses  are  performed. 

GAO  also  recommends  that  DOD 
require  program  offices  to  collect 
and  report  detailed  support  cost 
data  for  their  PBL  arrangements. 
DOD  generally  concurred  with 
these  recommendations. 

To  view  the  full  product,  including  the  scope 
and  methodology,  click  on  GAO-09-41. 

For  more  information,  contact  William  M.  Solis 
at  (202)  512-8365  orsolisw@gao.gov. 


Although  DOD’s  guidance  recommends  that  business  case  analyses  be  used  to 
guide  decision  making  regarding  the  implementation  of  PBL  to  provide 
weapon  system  support,  the  services  are  not  consistent  in  their  use  of  such 
analyses.  About  half  of  the  DOD  program  offices  responsible  for  the  29  PBL 
arrangements  GAO  reviewed  either  did  not  use  a  business  case  analysis  or 
could  not  provide  documentation  for  significant  parts  of  their  analyses. 

Almost  all  of  the  remaining  analyses  were  missing  one  or  more  of  the 
recommended  elements  in  DOD’s  instruction  for  economic  analysis.  Finally, 
business  case  analyses  were  often  not  updated  in  accordance  with  service 
policies  and  guidance.  Program  office  use  of  these  analyses  is  inconsistent 
because  DOD  only  recommends,  but  does  not  require,  that  they  be  prepared 
and  because  DOD’s  guidance  on  preparing  a  business  case  analysis  is  not 
comprehensive  and  does  not  adequately  specify  the  criteria  to  be  included. 
Also,  most  of  the  services  have  not  established  effective  internal  controls  to 
ensure  that  the  analyses  are  prepared  or  that  they  provide  a  consistent  and 
comprehensive  assessment.  As  a  result,  DOD  has  implemented  PBL 
arrangements  without  the  benefit  of  sound  analyses  that  ensure  that  the 
chosen  approach  will  provide  the  most  cost-effective  support  option. 

While  one  of  DOD’s  goals  in  moving  toward  the  use  of  PBL  arrangements  was 
to  reduce  weapon  system  support  costs,  the  ability  of  these  arrangements  to 
reduce  costs  remains  unclear  7  years  after  DOD  first  identified  PBL  as  the 
preferred  weapon  system  support  strategy.  Many  DOD  program  offices  that 
implemented  PBL  arrangements  have  limited  cost  data,  and  various  other 
factors — such  as  the  lack  of  business  case  analyses — further  limit  an 
evaluation  of  the  costs  of  this  support  strategy.  Available  data  from  the 
programs  GAO  reviewed  indicated  mixed  results.  Although  a  few  programs  in 
GAO’s  sample  provided  evidence  of  some  cost  reductions,  GAO’s  analysis  of 
the  only  two  systems  in  its  sample  that  are  managed  using  both  a  PBL 
arrangement  and  a  more  traditional,  non-PBL  arrangement  indicated  that  in 
both  cases  the  PBL  arrangement  had  higher  costs.  Also,  GAO  found  that 
certain  characteristics  of  DOD’s  PBL  arrangements — contract  length,  funding 
stability,  ownership  of  inventory,  and  the  lack  of  cost  metrics  and  effective 
incentives — could  limit  the  ability  of  and  incentive  for  contractors  to  reduce 
support  costs.  Neither  DOD  nor  the  services  require  detailed  cost  reporting 
for  PBL  arrangements  and  the  lack  of  detailed  cost  data  hinders  DOD’s  ability 
to  determine  whether  PBL  has  reduced  support  costs  as  intended. 

GAO  describes  the  use  of  performance-based  arrangements  for  weapon 
system  support  in  the  United  Kingdom’s  Ministry  of  Defence,  which  the 
ministry  refers  to  as  contracting  for  availability.  The  Ministry  of  Defence 
began  awarding  availability  contracts  as  an  approach  to  reduce  weapon 
system  support  costs,  and  officials  believe  that  support  cost  reductions  have 
been  achieved  as  a  result  of  using  availability  contracts.  In  general,  the 
availability  contracts  used  are  significantly  longer  than  those  used  by  DOD, 
and  the  ministry  uses  an  “open  book  accounting”  arrangement  to  gain 
visibility  into  the  contractors’  costs  to  provide  support. 
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United  States  Government  Accountability  Office 
Washington,  DC  20548 


December  19,  2008 

The  Honorable  Solomon  P.  Ortiz 
Chairman 

The  Honorable  J.  Randy  Forbes 
Ranking  Member 
Subcommittee  on  Readiness 
Committee  on  Armed  Services 
House  of  Representatives 

In  the  1990s,  the  Department  of  Defense  (DOD)  predicted  that  it  would 
save  billions  of  dollars  by  reengineering  defense  logistics  to  increase  its 
reliance  on  the  private  sector  for  the  support  of  military  weapon  systems.1 
In  1999,  DOD  set  a  goal  to  reduce  weapon  system  operating  and  support 
costs  by  20  percent  by  2005.  Around  the  same  time,  a  DOD  study  identified 
30  pilot  programs  on  which  to  test  various  logistics  reengineering 
initiatives,  and  DOD  began  to  consider  the  concept  of  long-term  contractor 
logistics  weapon  system  support  with  performance  metrics,  which  DOD 
later  called  performance  based  logistics  (PBL).  By  2001,  DOD  had 
identified  PBL  as  its  preferred  weapon  system  support  strategy.  DOD 
defines  PBL  as  the  purchase  of  performance  outcomes  (such  as  the 
availability  of  functioning  weapon  systems)  through  long-term  support 
arrangements  rather  than  the  purchase  of  individual  elements  of  support — 
such  as  parts,  repairs,  and  engineering  support.  The  concept  of  PBL 
evolved  from  the  more  general  concept  of  performance-based  contracting, 
which  has  been  used  in  both  the  commercial  and  government  sectors  for 
over  20  years.  While  the  use  of  PBL  was  first  applied  to  weapon  system 
platforms,  DOD  now  also  uses  it  to  purchase  support  for  subsystems  and 
components.  Under  DOD  guidance,2  prior  to  entering  into  a  PBL 


1  “Weapon  system  support”  includes  functions  such  as  materiel  management,  distribution, 
technical  data  management,  maintenance,  training,  cataloging,  configuration  management, 
engineering  support,  obsolescence  management,  technology  refreshment,  and  repair  parts 
management. 

2  Department  of  Defense,  Product  Support  for  the  21st  Century:  A  Program  Manager’s 
Guide  to  Buying  Performance  (Washington,  D.C.,  Nov.  6,  2001);  Under  Secretary  of 
Defense  (Acquisition,  Technology  and  Logistics),  Memorandum,  Performance  Based 
Logistics  (PBL)  Business  Case  Analysis  (BCA)  (Jan.  23,  2004);  Under  Secretary  of  Defense 
(Acquisition,  Technology  and  Logistics),  Memorandum,  Performance  Based  Logistics 
(PBL)  and  Business  Case  Analysis  (BCA)  (Mar.  20,  2004);  and  Department  of  Defense, 
Defense  Acquisition  University,  Performance  Based  Logistics:  A  Program  Manager’s 
Product  Support  Guide  (Fort  Belvoir,  Va.,  March  2005). 
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arrangement,  program  offices  should  conduct  a  business  case  analysis  as 
part  of  their  decision-making  process.  A  business  case  analysis  is  an 
analytical  tool  for  assessing  the  projected  costs  and  benefits  of  a  PBL 
arrangement  compared  with  other  alternative  weapon  system  support 
options.  However,  since  DOD  first  implemented  PBL  arrangements, 
questions  have  arisen  about  whether  they  have  reduced  the  costs  of 
weapon  system  support  as  intended. 

In  2004  and  2005,  we  reported  that  DOD’s  use  of  PBL  arrangements  did  not 
reflect  the  best  practices  of  private-sector  companies  and  that  DOD  could 
not  demonstrate  that  PBL  arrangements  had  reduced  costs  and  improved 
performance.3  Specifically,  in  2004,  we  recommended  that  DOD 
incorporate  into  guidance  the  private  sector’s  practice  of  using 
performance-based  contracts  at  the  component  level  (for  example,  to 
support  engines  or  auxiliary  power  units)  rather  than  only  at  the  weapon 
system  platform  level  (for  example,  to  support  aircraft).  In  response,  DOD 
issued  policy  memorandums  emphasizing  that  PBL  may  be  applied  at  the 
weapon  system,  subsystem,  or  component  level  and  revised  its  guidance  to 
implement  our  recommendation.  In  2005,  we  recommended  that  DOD,  in 
conjunction  with  the  military  services,  develop  procedures  to  track 
whether  DOD  program  offices  update  their  business  case  analyses  as 
called  for  under  DOD  guidance.  We  also  recommended  that  DOD  verify 
the  reliability  of  contractor  cost  and  performance  data.  DOD  concurred 
with  our  recommendations  and  has  issued  limited  guidance  to  the  services 
regarding  the  verification  of  the  reliability  of  contractor  cost  and 
performance  data.  However,  DOD  and  the  services,  with  the  exception  of 
the  Army,  have  not  developed  procedures  to  track  whether  program 
offices  update  business  case  analyses  after  PBL  implementation. 

Given  concerns  about  the  cost  of  programs  using  PBL  arrangements,  you 
requested  that  we  examine  DOD’s  use  of  PBL  for  providing  weapon 
system  support.  In  response  to  your  request,  we  evaluated  the  (1)  extent  to 
which  DOD  has  used  business  case  analyses  to  guide  decisions  related  to 
PBL  arrangements  and  (2)  impact  PBL  arrangements  have  had  on  weapon 
system  support  costs.  In  addition,  we  describe  the  use  and  characteristics 
of  “availability  contracts,”  a  concept  similar  to  PBL  arrangements  that  the 
United  Kingdom’s  Ministry  of  Defence  uses  for  weapon  system  support. 


3  GAO,  Defense  Management:  DOD  Needs  to  Demonstrate  That  Performance-Based 
Logistics  Contracts  Are  Achieving  Expected  Benefits,  GAO-05-966  (Washington,  D.C.: 
Sept.  9,  2005),  and  Defense  Management:  Opportunities  to  Enhance  the  Implementation 
of  Performance-Based  Logistics,  GAO-04-715  (Washington,  D.C.:  Aug.  16,  2004). 
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We  looked  at  the  Ministry  of  Defence’s  use  of  availability  contracts  to 
identify  potential  promising  practices  because  reducing  costs  was  a  clear 
goal  of  the  implementation  of  this  concept  in  the  United  Kingdom  and  a 
National  Audit  Office  report  stated  that  the  Ministry  of  Defence  has 
reduced  support  costs  as  a  result  of  its  implementation  of  this  concept, 
and  related  initiatives,  for  two  aircraft  systems. 

We  collected  and  analyzed  data  on  27  PBL  arrangements — specifically  9 
from  the  Army,  8  from  the  Navy,  and  10  from  the  Air  Force — that  were 
initiated  from  1996  through  2007  to  support  DOD  weapon  systems.4  In 
addition,  we  reviewed  2  additional  weapon  systems  that  the  Marine  Corps 
was  considering  as  candidates  for  PBL  arrangements.5  We  selected  these 
programs  from  lists  of  PBL  arrangements  provided  by  the  services  and 
chose  both  component-  and  system-level  arrangements  from  each  of  the 
services  on  the  basis  of  the  dollar  value  of  the  arrangements,  prior  audit 
findings,  and  location.6  While  we  focused  on  the  costs  of  PBL 
arrangements,  program  officials  said  that  performance  generally  met  or 
exceeded  requirements  established  in  the  contracts  or  other  agreements. 

To  evaluate  the  extent  to  which  DOD  has  used  business  case  analyses  to 
guide  decisions  regarding  PBL  arrangements,  we  obtained  and  examined 
the  analyses  prepared  by  program  offices.  In  assessing  the  quality  of  these 
analyses,  we  used  applicable  criteria  in  DOD  Instruction  7041.3,  Economic 
Analysis  for  Decisionmaking.7  This  economic  analysis  instruction  is 
generally  consistent  with  DOD’s  guidance  on  conducting  a  business  case 
analysis;  in  both  cases,  DOD  recommends  that  costs  and  benefits  (both 
qualitative  and  quantitative)  be  considered  and  documented.  However,  the 
economic  analysis  instruction  contains  more  specific  information  for 
evaluation  of  program  costs.  This  guidance  is  also  consistent  with  Office 


4  Of  the  29  PBL  arrangements,  6  were  first  initiated  before  2001  when  DOD  guidance 
recommended  conducting  a  business  case  analysis;  however,  4  of  the  6  negotiated 
subsequent  PBL  arrangements  after  2001.  The  remaining  2  programs  conducted  a  business 
case  analysis. 

5  According  to  Marine  Corps  officials,  no  PBL  arrangements  for  ground  systems  have  been 
implemented  yet.  The  Naval  Air  Systems  Command  manages  PBL  arrangements  for 
support  of  Marine  Corps  aviation  systems. 

6  The  results  from  this  nonprobability  sample  cannot  be  used  to  make  inferences  about  the 
population  because  the  sample  may  not  reflect  all  of  characteristics  of  the  population. 

'  DOD  Instruction  7041.3,  Economic  Analysis  for  Decisionmaking,  November  7,  1995.  In 
this  report,  we  refer  to  this  as  DOD’s  economic  analysis  instruction. 
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of  Management  and  Budget  guidelines  for  the  benefit-cost  analysis  of 
federal  programs.8  DOD’s  economic  analysis  instruction  also  states  that 
analytical  studies  that  evaluate  the  cost  and  effectiveness  of  weapon 
system  support  are  considered  to  be  “economic  analyses”  even  if  not  titled 
as  such. 

To  evaluate  the  impact  that  PBL  arrangements  have  had  on  weapon 
system  support  costs,  we  asked  program  officials  to  identify  and 
document  savings  attributed  to  PBL  implementation.  In  addition,  if  a 
program  had  renewed  a  PBL  arrangement  or  had  finalized  contract 
options  that  were  not  priced,  we  analyzed  the  contracts  for  trends  in  PBL 
support  costs.  We  also  compared  PBL  contract  costs  with  estimated  PBL 
support  costs  in  business  case  analyses  to  determine  how  closely  the 
estimates  matched  the  actual  PBL  arrangement  costs,  where  available.  We 
also  relied  on  our  previously  issued  reports  and  testimonies  on  DOD’s 
implementation  of  PBL. 

To  provide  information  regarding  the  characteristics  and  use  of  availability 
contracts  for  weapon  system  support  in  the  United  Kingdom’s  Ministry  of 
Defence,  we  met  Ministry  of  Defence  and  National  Audit  Office  officials 
and  officials  from  defense  contractors  having  availability  contracts  with 
the  Ministry  of  Defence.  We  also  reviewed  a  2007  National  Audit  Office 
report  regarding  the  Ministry  of  Defence’s  use  of  availability  contracting  to 
support  Harrier  and  Tornado  fast  jets.9  We  obtained  these  data  for 
informational  purposes  and  to  identify  promising  practices  and  did  not 
independently  verify  the  statements  or  data  provided  by  Ministry  of 
Defence  and  National  Audit  Office  officials. 

We  conducted  this  performance  audit  from  February  2007  through 
December  2008  in  accordance  with  generally  accepted  government 
auditing  standards.  Those  standards  require  that  we  plan  and  perform  the 
audit  to  obtain  sufficient,  appropriate  evidence  to  provide  a  reasonable 
basis  for  our  findings  and  conclusions  based  on  our  audit  objectives.  We 
believe  that  the  evidence  obtained  provides  a  reasonable  basis  for  our 
findings  and  conclusions  based  on  our  audit  objectives.  A  detailed 
description  of  our  scope  and  methodology  is  contained  in  appendix  I. 


8  Office  of  Management  and  Budget,  Circular  A-94,  Guidelines  and  Discount  Rates  for 
Benefit-Cost  Analysis  of  Federal  Programs  (Oct.  29,  1992). 

9  National  Audit  Office,  Transforming  Logistics  Support  for  Fast  Jets  (London:  July  17, 
2007). 
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Results  in  Brief 


Although  DOD’s  guidance  recommends  that  business  case  analyses  be 
used  to  guide  decision  making  regarding  the  implementation  of  PBL  to 
provide  weapon  system  support,  the  services  are  not  consistent  in  their 
use  of  such  analyses.  In  general,  for  the  PBL  arrangements  we  reviewed 
business  case  analyses  either  were  not  done,  were  not  fully  documented, 
or  were  not  comprehensive  or  sound.  About  half  of  the  DOD  program 
offices  responsible  for  the  29  PBL  arrangements  we  reviewed  either  did 
not  use  a  business  case  analysis  (or  economic  analysis)  or  could  not 
provide  documentation  for  significant  parts  of  their  analyses.  Almost  all  of 
the  remaining  analyses  were  missing  one  or  more  of  the  recommended 
elements  in  DOD’s  economic  analysis  instruction.  Finally,  business  case 
analyses  were  often  not  updated  in  accordance  with  service  policies  and 
guidance.  Program  office  use  of  these  analytical  tools  is  inconsistent 
because  (1)  DOD’s  guidance  did  not  require  that  they  be  prepared  and 
updated  and  (2)  DOD  did  not  provide  specific  criteria  for  conducting  the 
analyses.  Additionally,  most  of  the  services  have  not  established  effective 
internal  controls  to  ensure  that  the  analyses  are  prepared  or  that  they 
provide  a  consistent  and  comprehensive  assessment  of  weapon  system 
support  options.  As  a  result,  DOD  has  implemented  PBL  arrangements  to 
provide  weapon  system  support  without  sound  analyses  that  ensure  that 
the  chosen  approach  will  provide  the  most  cost-effective  support  option 
for  the  department. 

Although  the  PBL  concept  was  an  initiative  in  a  DOD  logistics 
reengineering  pilot  program  intended  to  reduce  weapon  system  support 
costs,  the  ability  of  these  arrangements  to  reduce  costs  remains  unclear  7 
years  after  DOD  first  identified  PBL  as  the  preferred  support  strategy. 
Many  DOD  program  offices  that  implemented  PBL  arrangements  have 
limited  cost  data,  and  various  other  factors,  such  as  the  lack  of  business 
case  analyses  and  changing  operational  and  materiel  conditions,  further 
limit  an  evaluation  of  the  costs  of  this  support  strategy.  Available  data 
from  the  programs  we  reviewed  indicated  mixed  results.  Although  a  few 
programs  in  our  sample  provided  evidence  of  some  cost  reductions,  our 
analysis  of  the  only  two  systems  in  our  sample  that  were  managed  using 
both  a  PBL  arrangement  and  a  more  traditional,  non-PBL  arrangement 
indicated  that  the  PBL  arrangement  had  higher  costs  in  both  cases.  We 
also  found  few  examples  of  decreasing  support  costs  after  PBL  contract 
renewal,  as  the  PBL  concept  suggests  should  occur.  Finally,  we  found  that 
certain  characteristics  of  DOD’s  PBL  arrangements — such  as  short-term 
contracts,  unstable  requirements  and  funding,  DOD  ownership  of 
inventory,  and  the  lack  of  cost  metrics  and  effective  incentives — could 
limit  the  ability  of  and  incentive  for  contractors  to  reduce  support  costs. 
Neither  DOD  nor  the  services  require  detailed  cost  reporting  for  PBL  (or 
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for  any  type  of  contractor  logistics  support).  Further,  in  its  implementation 
of  PBL  arrangements,  DOD  has  emphasized  the  potential  to  improve 
performance  and  not  the  potential  to  reduce  support  costs.  The  lack  of 
detailed  cost  data  for  evaluating  the  impact  of  PBL  arrangements  on 
support  costs,  compounded  with  deficiencies  in  DOD’s  use  of  business 
case  analyses  for  decision  making,  hinders  DOD’s  ability  to  determine 
whether  most  PBL  arrangements  have  reduced  support  costs  as  intended. 

We  describe  the  use  of  performance-based  arrangements  for  weapon 
system  support  in  the  United  Kingdom’s  Ministry  of  Defence,  which  is 
referred  to  as  contracting  for  availability.  The  Ministry  of  Defence  began 
awarding  availability  contracts — generally  for  aviation  systems  but  also  for 
some  maritime  equipment — as  an  approach  to  reduce  weapon  system 
support  costs  after  a  1999  cost  reduction  goal  of  20  percent  was  set. 
Ministry  of  Defence  officials  believe  that  support  cost  reductions  have 
been  achieved  as  a  result  of  using  availability  contracts,  and  the  National 
Audit  Office  reported  savings  for  the  Tornado  and  Harrier  fast  jets, 
although  it  is  unclear  to  what  extent  the  availability  contracts  generated 
the  cost  reductions  because  other  cost  reduction  initiatives  were 
implemented  at  the  same  time.  In  general,  the  availability  contracts  used 
by  the  ministry  are  significantly  longer  than  those  used  by  DOD,  and  the 
ministry  uses  an  “open  book  accounting”  arrangement  to  gain  visibility 
into  the  contractors’  costs  to  provide  support. 

We  are  making  recommendations  in  this  report  aimed  at  documenting  and 
improving  the  analysis  of  the  cost-effectiveness  of  PBL  arrangements.  We 
recommend,  for  example,  that  DOD  require  the  development  of  business 
case  analyses,  specifying  the  elements  to  be  included  in  them  so  that  they 
are  comprehensive  and  sound  and  that  the  services  improve  their  internal 
controls  to  ensure  that  the  analyses  are  performed,  and  that  program 
offices  collect  and  report  detailed  support  cost  data  for  their  PBL 
arrangements.  Improvements  in  the  collection  and  reporting  of  detailed 
cost  data  could  go  a  long  way  toward  providing  required  information  for 
determining  if  PBL  provides  the  most  cost-effective  approach  for 
supporting  DOD  weapon  systems.  In  written  comments  on  a  draft  of  this 
report,  DOD  generally  concurred  with  our  recommendations  and 
discussed  actions  it  has  planned  to  implement  some  of  our 
recommendations,  such  as  revising  DOD’s  acquisition  directive  to  require 
the  development  of  a  business  case  analysis.  DOD  also  stated  that  it  would 
review  existing  policy  and  guidance  in  other  areas  as  part  of  an  ongoing 
study.  While  we  view  DOD’s  actions  as  positive  steps  to  improve  the 
implementation  of  PBL,  it  is  unclear  to  what  extent  the  results  of  the 
study,  and  any  related  actions,  will  affect  the  evaluation  of  the  cost- 
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effectiveness  of  DOD’s  PBL  arrangements.  DOD’s  comments  are  reprinted 
in  appendix  II. 


Background 


PBL  is  a  method  of  providing  support  for  weapon  systems  by  designating 
what  system  performance  is  required,  such  as  a  given  level  of  system 
availability,  and  placing  the  responsibility  for  how  it  is  accomplished  on 
the  support  provider,  which  manages  resources  to  achieve  performance 
objectives.  Logistics  support  for  almost  all  of  DOD’s  weapon  systems,  such 
as  materiel  management,  maintenance,  and  engineering,  is  provided  by  a 
combination  of  government  and  private-sector  sources.  In  the  past,  under 
traditional  support  arrangements,  the  government  generally  managed  the 
provision  of  weapon  system  support,  using  a  combination  of  support 
providers  from  the  government  and  the  private  sector.  PBL  support 
arrangements  often  use  a  private-sector  support  integrator  to  manage 
support  providers  from  both  the  public  and  private  sectors  to  meet 
specified  performance  requirements. 

PBL  evolved  from  performance-based  service  contracting,  which  has  been 
used  in  both  the  public  and  private  sectors.  The  Federal  Acquisition 
Regulation  defines  performance-based  contracting  as  structuring  all 
aspects  of  an  acquisition  around  the  purpose  of  the  work  to  be  performed. 
The  Federal  Acquisition  Regulation  further  defines  the  statement  of  work 
for  a  performance-based  acquisition  as  describing  the  required  results  in 
clear,  specific,  and  objective  terms  with  measurable  outcomes.10 
Performance-based  service  contracting  has  been  referenced  in  regulation, 
guidance,  and  policy  for  more  than  two  decades,  and  federal  agencies  have 
used  it  to  varying  degrees  for  acquiring  a  range  of  services.  In  1991  the 
Office  of  Management  and  Budget  issued  a  policy  letter  establishing  the 
use  of  a  performance-based  approach  for  service  contracting,  and  in  1994 
it  initiated  a  govemmentwide  pilot  project  to  encourage  the  use  of 
performance-based  service  contracts  in  federal  agencies,  including  DOD. 

In  October  1997,  the  Federal  Acquisition  Regula  tion  was  amended  to 
incorporate  the  Office  of  Management  and  Budget’s  1991  policy.  The 
Federal  Acquisition  Regula  tion  currently  establishes  a  policy  that 


10  Federal  Acquisition  Regulation  Subpart  2.101. 
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agencies  use  performance-based  contracting  methods  to  the  maximum 
extent  practicable  for  the  acquisition  of  services,  with  certain  exceptions.11 

Using  performance-based  service  contracts  is  intended  to  offer  a  number 
of  potential  benefits,  such  as  encouraging  contractors  to  be  innovative  and 
to  find  cost-effective  ways  of  delivering  services  for  a  fixed  level  of 
funding.  By  shifting  the  focus  from  process  to  results,  these  contracts  can 
potentially  produce  better  outcomes  and  reduced  costs.  The  Office  of 
Management  and  Budget  reported  that  the  agencies  participating  in  the 
pilot  reduced  contract  prices  and  improved  customer  satisfaction  with 
contractor  work  after  introducing  performance-based  contracting. 


PBL  Implemented  as  a 
Strategy  to  Reduce  or 
Minimize  Costs  and 
Improve  Performance 


As  an  approach  for  supporting  military  weapon  systems,  PBL  emerged 
from  a  1999  DOD  study  to  test  logistics  reengineering  concepts  that  placed 
greater  reliance  on  the  private  sector  for  providing  weapon  system 
support  to  reduce  support  costs  and  improve  weapon  system 
performance.  The  goal  was  for  the  military  departments  to  reduce  costs 
and  improve  efficiency  by  pursuing  logistics  support  “reengineering” 
efforts  using  contractors.12  The  fiscal  years  2001-2005  Defense  Planning 
Guidance  advanced  this  cost  reduction  effort  by  establishing  a  goal  that 
directed  each  military  department  to  reduce  the  operation  and  support 
costs  of  its  fielded  systems  by  20  percent  by  the  year  2005.  During  this 
time,  the  Under  Secretary  of  Defense  (Acquisition  and  Technology) 
directed  the  services  to  use  an  existing  pilot  program  containing  30 
weapon  systems  to  demonstrate  the  type  of  cost  savings  depicted  in  the 
fiscal  years  2001-2005  Defense  Planning  Guidance.  The  areas  identified  for 
potential  cost  savings  were  reducing  demand  on  the  supply  chain  by 
improving  the  reliability  and  maintainability  of  the  equipment,  reducing 
supply  chain  response  time,  and  increasing  competitive  sourcing  of 
product  support.  Some  of  the  30  pilot  programs  involved  performance- 
type  arrangements  that  the  services  subsequently  converted  to,  or 
designated  as,  PBL  arrangements. 


11  Federal  Acquisition  Regulation  Subpart  37. 102(a),  implementing  section  821  of  the 
Floyd  D.  Spence  Defense  Authorization  Act  for  Fiscal  Year  2001  (Pub.  L.  No.  106-398).  The 
Federal  Acquisition  Regulation  addresses  performance-based  contracting  generally  at 
Subpart  37.6. 

12  GAO,  Defense  Logistics:  Actions  Needed  to  Enhance  Success  of  Reengineering 
Initiatives,  GAO/NSIAD-OO-89  (Washington,  D.C.:  June  23,  2000). 
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This  emphasis  on  reducing  costs  through  PBL  implementation  was 
reiterated  in  DOD’s  2001  Quadrennial  Defense  Review  Report,13  which 
advocated  the  implementation  of  PBL  to  compress  the  supply  chain  by 
removing  steps  in  the  warehousing,  distribution,  and  order  fulfillment 
processes;  to  reduce  inventories;  and  to  decrease  overhead  costs  while 
improving  the  readiness  of  major  weapon  systems  and  commodities.  In 
November  2001,  DOD  identified  PBL  as  the  preferred  weapons  system 
support  strategy.14  In  May  2003,  DOD  further  strengthened  this  emphasis 
on  PBL  by  stating  in  a  DOD  policy  directive  that  acquisition  managers  shall 
use  performance-based  strategies  for  sustaining  products  and  services 
whenever  feasible  and  PBL  strategies  shall  optimize  total  system 
availability  while  minimizing  cost  and  the  logistics  footprint.15 

In  concept,  a  properly  structured  PBL  arrangement  is  supposed  to  provide 
a  level  of  performance  and  also  reduce  costs  over  time.  According  to  the 
DOD/Defense  Acquisition  University  PBL  guide,16  a  key  aspect  of  PBL  is 
the  inclusion  of  an  incentive  for  the  support  provider  to  reduce  costs 
through  increased  reliability.  Further,  PBL  arrangements  can  inherently 
motivate  support  providers  to  improve  component  and  system  reliability, 
since  such  improvements  can  provide  the  foundation  for  increased  profit 
over  the  long  term.  In  other  words,  the  support  provider  should  have  the 
incentive  to  make  reliability  improvements  to  ensure  that  performance 
metrics  are  met  and  also  to  increase  profit  by  earning  a  performance 


13  Department  of  Defense,  Quadrennial  Defense  Review  Report  (Washington,  D.C., 

Sept.  30,  2001). 

14  Department  of  Defense,  Product  Support  for  the  21st  Century:  A  Program  Manager’s 
Guide  to  Buying  Performance.  Throughout  this  report,  we  refer  to  this  document  as 
DOD’s  2001  PBL  guide. 

15  DOD  Directive  5000.1,  The  Defense  Acquisition  System,  paras.  El.  1.16  and  El.  1.17, 

May  12,  2003.  The  logistics  footprint  refers  to  the  size  or  presence  of  government  or 
contractor  personnel,  equipment,  inventory,  facilities,  transportation  assets,  or  real  estate 
needed  to  deploy,  sustain,  and  move  the  weapon  systems  or  components. 

16  Department  of  Defense,  Defense  Acquisition  University,  Performance  Based  Logistics:  A 
Program.  Manager’s  Product  Support  Guide  (Washington,  D.C.,  Sept.  30,  2004).  This 
document,  which  was  reissued  in  March  2005,  supersedes  DOD’s  Product  Support  for  the 
21st.  Century:  A  Program,  Manager’s  Guide  to  Buying  Performance,  dated  November 
2001.  In  a  November  2004  memorandum,  the  Acting  Under  Secretary  of  Defense 
(Acquisition,  Technology  and  Logistics)  described  the  Defense  Acquisition  University’s 
Program,  Manager’s  Product  Support  Guide  as  guidance  that  should  be  used  by  DOD 
program  and  product  support  managers  in  conjunction  with  other  DOD  policy  and  Office  of 
the  Secretary  of  Defense  memorandums.  Throughout  this  report,  we  refer  to  this  guide  as 
the  DOD/Defense  Acquisition  University  PBL  guide. 
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incentive  tied  to  the  metrics  (an  award  fee  or  award  term)  and  by  reducing 
costs  while  still  being  paid  the  agreed-upon,  fixed  price  for  the  remainder 
of  the  contract.  The  DOD/Defense  Acquisition  University  PBL  guide  also 
states  that  a  critical  element  of  the  PBL  arrangement  that  facilitates  this 
incentive  and  motivation  is  contract  length.  Further,  long-term  contracts 
provide  the  support  provider  with  confidence  in  continuing  cash  flows  and 
provide  sufficient  time  for  receiving  an  adequate  return  on  any 
investments  made  to  improve  reliability. 


DOD  Recommended  That 
Analyses  Be  Conducted  to 
Guide  Decisions  regarding 
Weapon  System  Support 
Options 


In  1995,  before  DOD  identified  PBL  as  the  preferred  weapons  system 
support  strategy,  DOD’s  economic  analysis  instruction  recommended 
using  an  economic  analysis  for  evaluating  options  for  weapon  system 
support.  The  instruction  stressed  the  importance  of  considering  in  the 
analysis  both  qualitative  and  quantitative  factors.  With  respect  to 
quantitative  factors,  the  instruction  recommended  that  costs  and  benefits 
be  expressed  in  terms  of  net  present  value  to  account  for  the  time  value  of 
money.  These  were  also  to  be  expressed  as  life  cycle  costs  and  benefits 
that  were  to  be  calculated  and  compared  for  each  feasible  alternative  for 
meeting  a  given  weapon  system  support  objective. 


Specifically,  the  economic  analysis  instruction  identified  and 
characterized  the  following  seven  elements  that  should  be  present  in  an 
economic  analysis: 


•  objectives — to  clearly  identify  the  function  to  be  accomplished  and  not 
to  assume  a  specific  means  of  achieving  a  desired  result; 

•  assumptions — to  incorporate  both  actual  data  and  future 
uncertainties; 

•  alternatives — to  comprise  a  comprehensive  list  of  the  feasible  and 
infeasible  options  followed  by  a  discussion  of  the  infeasible  options 
and  comparisons  of  only  the  feasible  options; 

•  costs  and  benefits — to  compare  the  quantitative  (expressed  in  terms  of 
net  present  value)  and  qualitative  factors  for  each  option; 

•  sensitivity  and  uncertainty  (risk)  analyses — to  determine  the  effect 
of  uncertainties  on  the  results  of  the  analysis  and  to  provide  a  range  of 
costs  and  benefits; 

•  summary  of  the  results  of  the  analysis-,  and 

•  summary  of  the  analysis's  recommendations. 
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In  DOD’s  2001  PBL  guide,  in  which  DOD  identified  PBL  as  the  preferred 
weapon  system  support  strategy,  the  department  recommended  that  for  all 
new  systems  and  fielded  acquisition  category  I  and  II  systems,17  program 
offices  use  an  analytical  tool  called  a  business  case  analysis  to  support  the 
decision  to  use  PBL  arrangements  for  weapon  system  support. 

In  2004  and  2005,  DOD  guidance  on  conducting  business  case  analyses 
described  this  tool  in  less  specific  terms  than  those  used  to  describe  the 
criteria  laid  out  in  DOD’s  economic  analysis  instruction.  However,  there 
are  some  common  themes  in  the  guidance  and  instruction.  For  example,  a 
January  2004  Under  Secretary  of  Defense  (Acquisition,  Technology  and 
Logistics)  memorandum  on  PBL  business  case  analysis  calls  for  an 
assessment  of  “best  value,”  or  the  greatest  overall  benefit.18  The 
DOD/Defense  Acquisition  University  PBL  guide  repeats  the 
characterization  of  the  business  case  analysis  as  a  tool  with  the  goal  of 
determining  a  best-value  solution  and  suggests  that  at  a  minimum  a 
business  case  analysis  should  include  an  introduction  outlining  the 
purpose  and  objectives  of  the  program,  an  explanation  of  the  methods  and 
assumptions  used  in  the  analysis,  calculations  of  the  relative  costs  and 
benefits  of  each  weapon  system  support  option,  the  financial  and 
nonfinancial  impacts  of  each,  a  risk  assessment,  and  a  section  with 
conclusions  and  recommendations.  Finally,  both  DOD’s  economic  analysis 
instruction  and  the  DOD/Defense  Acquisition  University  PBL  guide 
recommend  documenting  the  results  of  the  analysis,  including  all 
calculations  and  sources  of  data,  down  to  the  most  basic  inputs,  to  provide 
an  auditable  and  stand-alone  document.  According  to  the  guidance,  a 
business  case  analysis  must  stand  on  its  own  and  be  able  to  withstand 
rigorous  analysis  and  review  by  independent  agencies. 


17  The  acquisition  category  determines  the  level  of  review,  decision  authority,  and 
applicable  procedures  for  an  acquisition  program.  Acquisition  category  I  programs  have  an 
estimated  total  of  expenditure  of  more  than  $365  million  for  research,  development,  test, 
and  evaluation  or  procurement  of  more  than  $2.19  billion.  Acquisition  category  II  programs 
have  an  estimated  total  expenditure  for  research,  development,  test,  and  evaluation  from 
$140  million  to  $365  million  or  procurement  from  $660  million  to  $2.19  billion. 

18  Under  Secretary  of  Defense  (Acquisition,  Technology  and  Logistics),  Memorandum 
(Jan.  23,  2004).  Throughout  this  report  we  refer  to  this  as  the  January  2004  Under  Secretary 
of  Defense  (Acquisition,  Technology  and  Logistics)  memorandum. 
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DOD’s  2004  and  2005  guidance  on  conducting  business  case  analyses  also 
recommended  that  program  offices 

•  update  their  business  case  analyses  at  key  decision  points  both  to 
validate  the  approach  taken  and  to  support  future  plans  and 

•  use  certain  criteria,  such  as  the  capability  of  a  PBL  arrangement  to 
reduce  the  cost  per  operational  unit  of  performance  (i.e.,  cost  per 
flight  hour),  to  assess  all  acquisition  I  and  II  programs  without  plans 
for  a  PBL  arrangement  for  the  potential  application  of  a  PBL  strategy 
at  the  system,  subsystem,  or  major  assembly  level.  If  the  assessment 
showed  potential  for  a  PBL  arrangement,  a  business  case  analysis 
should  be  conducted  and  completed  by  the  September  30,  2006, 
deadline  required  by  DOD’s  Strategic  Planning  Guidance. 


PBL  Arrangements  In  our  review,  we  looked  at  PBL  arrangements  initiated  as  early  as  1996 

Included  in  Our  Review  (when  performance-based  contracting  was  encouraged  govemmentwide) 

and  as  recently  as  2007  (by  which  time,  at  the  DOD  level,  PBL 
arrangements  had  moved  from  being  encouraged  to  being  required 
whenever  feasible).  These  PBL  arrangements  represent  contract  values 
totaling  approximately  $12.1  billion  and  range  from  a  low  of  $10.5  million 
to  a  high  of  $4.9  billion.19  Table  1  lists,  by  service,  the  weapon  system 
programs  supported  by  the  29  PBL  arrangements  we  reviewed. 


19  The  costs  for  the  two  Marine  Corps  PBL  arrangements  are  not  included  in  this  estimate 
because  the  PBL  arrangements  have  not  yet  been  implemented.  Also,  the  total  does  not 
include  support  costs  for  the  Army’s  Common  Ground  Station  PBL  arrangement  because 
this  arrangement  is  with  a  government  activity  and  does  not  involve  a  contract.  According 
to  Common  Ground  Station  officials,  support  costs  for  fiscal  years  2003  through  2007 
totaled  about  $44  million. 
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Table  1 :  Weapon  System  Programs  Supported  by  PBL  Arrangements 


Service  Weapon  system 

Army  Tube-launched  Optically-tracked  Wire-guided  missile  -  Improved 

Target  Acquisition  System 

Javelin  antitank  missile 
High  Mobility  Artillery  Rocket  System 
RQ-7B  Shadow  Tactical  Unmanned  Aircraft  System 
Sentinel  Radar 
Patriot  Air  Defense  System 
AH-64D  Apache  Helicopter  -  Sensors 
AH-64D  Apache  Helicopter  -  Airframe 
AN/TSQ-179AV(2)  Common  Ground  Station 
Navy  F-1 6  engines 

KC-130J" 

V-22  engine 
T-45  engine 

Phalanx  Close-In  Weapon  System 
AV-8B  Harrier 
F/A-18  E/F 

Consolidated  Automated  Support  System 
Marine  Corps  Expeditionary  Fighting  Vehicle6 

Assault  Breacher  Vehicle6 
Air  Force  C-17  Globemaster  III 

T-6A  Joint  Primary  Air  Training  System 

F-117A  Nighthawk 
F-22  Raptor 
B-2  Spirit 

Secondary  Power  Systems 

E8-C  Joint  Surveillance  Target  Attack  Radar  System 
LITENING  Advanced  Airborne  Targeting  and  Navigation  Pod 
Sniper  Advanced  Targeting  Pod 
C-130J  Hercules 


Source:  GAO  analysis  of  DOD  data. 

“Although  the  KC-130J  is  a  Marine  Corps  aircraft,  the  support  contracts  are  managed  by  the  Navy. 
bA  PBL  arrangement  has  not  yet  been  implemented. 
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Business  Case 
Analyses  Have  Not 
Been  Used 
Consistently  or 
Effectively  to 
Influence  Decision 
Making  regarding  Use 
of  PBL 


DOD  has  generally  not  used  business  case  analyses  consistently  or 
effectively  to  influence  decision  making  regarding  the  use  of  PBL. 

Although  DOD  guidance  recommended  that  these  analyses  be  used  to 
guide  decisions  on  the  cost-effectiveness  of  weapon  system  support 
arrangements,  about  half  of  the  programs  we  reviewed  either  did  not 
conduct  such  an  analysis  or  did  not  retain  adequate  supporting 
documentation.20  Further,  most  of  the  remaining  programs  in  our  sample 
used  analyses  that  were  not  comprehensive.  For  example,  some  analyses 
did  not  evaluate  alternative  support  options  and  most  did  not  contain  all  of 
the  elements  recommended  in  DOD’s  economic  analysis  instruction. 
Additionally,  analyses  were  often  not  updated  to  support  decision  making 
after  PBL  implementation  in  accordance  with  service  policies  and 
guidance.  The  key  reasons  for  DOD’s  ineffective  use  of  business  case 
analyses  to  support  PBL  decision  making  are  that  (1)  DOD  has  not 
required  such  analyses  or  provided  specific  criteria  for  conducting  them 
and  (2)  the  services’  internal  controls  have  been  inadequate  to  ensure  that 
the  analyses  are  performed  and  updated.  As  a  result,  DOD  cannot  ensure 
that  decisions  regarding  weapon  system  support  options  are  guided  by 
comprehensive,  consistent,  and  sound  analytical  tools.  Further,  the 
department  cannot  be  sure  that  the  support  arrangements  being 
implemented  will  result  in  the  most  cost-effective  support  program. 


About  Half  of  Programs 
Reviewed  Either  Did  Not 
Use  Business  Case 
Analyses  to  Guide 
Decisions  to  Enter  into 
PBL  Arrangements  or  Did 
Not  Retain  Details  of  the 
Analysis 


For  9  of  the  29  PBL  arrangements  we  reviewed,  a  business  case  analysis 
had  not  been  completed.  Additionally,  for  6  others,  program  officials  could 
not  provide  supporting  details  of  the  business  case  analysis  they  told  us 
that  they  had  conducted.  When  program  offices  did  not  conduct  business 
case  analyses  as  part  of  their  PBL  decision  making,  as  we  found  for  many 
of  the  programs  we  reviewed,  the  department  cannot  be  sure  that  the 
support  arrangements  implemented  will  result  in  the  most  cost-effective 
support  program. 


Programs  That  Did  Not  Table  2  provides  the  number  of  PBL  arrangements  we  reviewed  by  service 

Conduct  a  Business  Case  that  were  initiated  with  and  without  the  use  of  a  business  case  analysis. 

Analysis  Before  Entering  into 
PBL  Arrangements 


20  According  to  Office  of  Management  and  Budget  Circular  A-94,  a  program  is  cost  effective 
if,  on  the  basis  of  a  life  cycle  cost  analysis  of  competing  alternatives,  it  is  determined  to 
have  the  lowest  costs  expressed  in  present  value  tenns  for  a  given  level  of  benefits. 
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Table  2:  Number  of  PBL  Arrangements  Initiated  with  and  without  the  Use  of  a 
Business  Case  Analysis 

Service 

Prepared 
an  analysis 

Did  not  prepare 
an  analysis 

Total 

Army 

7 

2 

9 

Navy 

6 

2 

8 

Marine  Corps 

2 

0 

2 

Air  Force 

5 

5 

10 

Total 

20 

9 

29 

Source:  GAO  analysis  of  DOD  data. 


Although  both  of  the  Marine  Corps  programs  we  reviewed  conducted  a 
business  case  analysis,  about  50  percent  of  Air  Force  programs,  22  percent 
of  Army  programs,  and  25  percent  of  Navy  programs  did  not.  In  general, 
the  Air  Force  programs  only  developed  a  source-of-repair  analysis,  which 
evaluated  only  the  repair  element  of  weapon  system  support.  For  the  two 
PBL  arrangements  for  which  the  Army  did  not  conduct  an  analysis — the 
Apache  sensor  and  airframe  arrangements — the  Deputy  Assistant 
Secretary  of  the  Army  (Integrated  Logistics  Support)  approved  the  Apache 
program  office’s  request  for  a  waiver  from  Army  business  case  analysis 
policy  based  on  prior  analyses  and  program  decisions.  However,  the  U.S. 
Army  Audit  Agency  reported  that  the  prior  analyses  did  not  consider  all 
components  included  in  the  two  PBL  arrangements,  other  support 
strategies,  performance  metrics,  and  relative  costs.21  According  to  an  Army 
official,  a  business  case  analysis  for  the  airframe  components  is  being 
conducted  and  is  expected  to  be  completed  in  December  2008,  and  efforts 
to  develop  a  business  case  analysis  for  the  program’s  sensors  are  expected 
to  begin  in  November  2008. 

The  F-22A  Raptor  and  KC-130J  are  examples  of  programs  where  the  Air 
Force  and  Navy  did  not  conduct  a  business  case  analysis  as  part  of  the 
PBL  decision-making  process.  When  DOD  recommended  in  2001  that 
program  offices  fielding  new  systems  base  PBL  arrangement  decisions  on 
business  case  analyses,  the  F-22  was  beginning  low-rate  initial  production. 
In  2002,  the  Assistant  Secretary  of  the  Air  Force  (Acquisition)  and  the  Air 
Force  Deputy  Chief  of  Staff  (Logistics,  Installations  and  Mission  Support) 


21  U.S.  Army  Audit  Agency,  Army’s  Process  and  Controls  for  Effectively  Implementing 
Performance  Based  Logistics,  A-2007-0072-ALM  (Alexandria,  Va.,  Feb.  9,  2007). 
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directed  the  F-22  program  office  to  develop  a  long-term  support  strategy 
and  manage  life  cycle  product  support  through  a  PBL  arrangement  that 
includes  government-contractor  partnerships  as  necessary.  From  2003 
through  2007,  the  program  office  acquired  support  as  part  of  the  aircraft’s 
production  contract,  and  in  2008  the  office  signed  a  separate  PBL  support 
contract  with  Lockheed  Martin,  one  of  the  original  equipment 
manufacturers,  to  support  the  aircraft  from  2008  to  2012.  In  March  2008, 
program  officials  said  that  they  did  not  conduct  a  business  case  analysis 
before  awarding  the  2008  contract  because  current  program  data,  such  as 
material  usage  rates  and  costs,  are  immature.  Officials  planned  to  conduct 
an  analysis  in  2010  or  2011  when  it  could  be  completed  using  more 
meaningful  data.  However,  program  officials  subsequently  decided  that  the 
available  data  were  sufficient  and  in  July  2008  awarded  a  contract  to 
develop  a  business  case  analysis.  Completion  of  the  analysis  is  expected  in 
late  2009. 

In  2002,  the  Navy  contracted  for  a  PBL  arrangement  to  support  the  Marine 
Corps’  KC-130J  engines  without  first  preparing  a  business  case  analysis. 
Program  officials  explained  that  the  decision  was  made  not  to  develop  an 
analysis  because  the  technical  data  needed  to  repair  the  engines  were  not 
available  and  the  Marine  Corps  did  not  have  the  infrastructure  necessary 
to  support  the  system.  Officials  also  said  that  a  market  analysis  was 
conducted  prior  to  implementing  the  PBL  arrangement,  but  they  could  not 
provide  a  copy  of  the  analysis.  Nonetheless,  a  business  case  analysis  that 
documented  the  results  of  the  market  analysis  and  the  program’s 
negotiated  costs  versus  expected  flight  hours  and  anticipated  repairs  and 
maintenance  could  have  been  used  to  monitor  the  actual  results  and  cost- 
effectiveness  of  the  performance-based  approach,  especially  since  support 
for  the  engines  is  obtained  under  a  commercial  contract  and  the 
contractor  does  not  provide  detailed  cost  data  to  the  government. 


Supporting 
Documentation  Not 
Fully  Available  for 
Some  Programs  That 
Conducted  Business 
Case  Analyses 


Program  officials  from  20  of  the  PBL  arrangements  we  reviewed  told  us 
that  they  had  conducted  business  case  analyses  before  implementing  the 
arrangements;  however,  officials  for  6  programs  could  not  provide  all  or 
some  of  the  data  to  support  the  analyses.  According  to  DOD’s  economic 
analysis  instruction,  the  results  of  the  analysis,  including  all  calculations 
and  sources  of  data,  must  be  documented  down  to  the  most  basic  inputs  to 
provide  an  auditable  and  stand-alone  document.  Table  3  lists  by  service 
the  number  of  PBL  arrangements  for  which  all  of  the  business  case 
analysis  documentation  was  retained  and  those  for  which  it  was  not. 
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Table  3:  Number  of  PBL  Arrangements  with  and  without  Supporting  Documentation 
of  a  Business  Case  Analysis 

Service 

Programs  that  did  retain 
supporting  documentation 

Programs  that 
did  not  retain  all  or  some 
supporting  documentation 

Total 

Army 

3 

4 

7 

Navy 

5 

1 

6 

Marine  Corps 

2 

0 

2 

Air  Force 

4 

1 

5 

Total 

14 

6 

20 

Source:  GAO  analysis  of  DOD  data. 


In  general,  program  officials  for  six  programs  said  that  they  were  unable  to 
locate  all  the  details  of  their  analyses;  however,  the  amount  of  data  that 
was  missing  varied.  For  example: 

•  Although  officials  for  the  Army’s  Common  Ground  Station  said  that  an 
analysis  was  performed  in  2002,  they  were  unable  to  provide  any 
details  of  the  analysis  or  the  results. 

•  While  program  officials  for  the  Army’s  Shadow  Tactical  Unmanned 
Aircraft  System  were  able  to  provide  the  results  of  their  2002  analysis, 
they  did  not  retain  the  details  regarding  the  assumptions,  data  sources, 
or  calculations  used  to  develop  the  analysis.  However,  program 
officials  said  that  the  analysis  was  developed  early  in  the  life  cycle  of 
the  program  and  was  not  based  on  historical  cost  and  maintenance 
data,  and  therefore  they  did  not  consider  it  to  be  very  accurate  based 
on  actual  program  results  that  have  occurred  since  the  analysis  was 
developed. 

•  For  the  Army’s  Javelin  PBL  arrangement,  the  DOD  Office  of  the 
Inspector  General  reported  in  2005  that  it  was  unable  to  validate  the 
program  office’s  2001  analysis  because  the  program  office  was  not  able 
to  provide  adequate  documentation.22  The  documentation  has  not  been 
located,  and  program  officials  were  only  able  to  provide  a  summary  of 
the  results  of  the  analysis. 


22  Department  of  Defense,  Office  of  the  Inspector  General,  Logistics:  Implementation  of 
Performance-Based  Logistics  for  the  Javelin  Weapon  System  .,  D-2005-037  (Arlington,  Va., 
Mar.  7,  2005). 
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Most  Remaining 
Programs  Used 
Business  Case 
Analyses  That  Were 
Not  Comprehensive 


Service  program  officials  could  provide  documentation  of  the  business 
case  analyses  conducted  for  14  of  the  PBL  arrangements  we  reviewed,  but 
all  but  1  of  the  14  analyses  were  missing  one  or  more  of  the  elements 
recommended  in  DOD’s  economic  analysis  instruction.  As  a  result, 
decisions  regarding  weapon  system  support  options  for  many  of  the 
programs  we  reviewed  were  not  guided  by  comprehensive,  consistent,  and 
sound  economic  analysis.  Further,  the  department  cannot  be  sure  that  the 
support  arrangements  implemented  will  result  in  the  most  cost-effective 
support  programs.  Figure  1  shows  which  elements  were  missing  from  the 
14  business  case  analyses. 
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Figure  1:  Elements  Missing  from  the  Business  Case  Analyses  We  Reviewed 


Service 


Weapon  system 


Army 

Tube-launched  Optically-tracked 
Wire-guided  missile — Improved 

Target  Acquisition  System 

A 

A 

® 

A 

A 

Sentinel  Radar 

® 

A 

Patriot  Air  Defense  System 

A 

© 

© 

Navy 

V-22  engine 

A 

A 

© 

A 

© 

Phalanx  Close-In  Weapon  System 

A 

A 

A 

A 

© 

A 

A 

AV-8B  Harrier 

A 

A 

A 

A 

© 

A 

A 

F/A-18  E/F  © 

A 

A 

A 

Consolidated  Automated  Support 
System 

A 

A 

A 

© 

A 

A 

Marine 

Corps 

Expeditionary  Fighting  Vehicle 

® 

Assault  Breacher  Vehicle 

A 

® 

Air  Force 

C-130J  Hercules 

C-17  Globemaster  III 

® 

© 

© 

B-2  Spirit 

A 

A 

® 

© 

© 

© 

Joint  Surveillance  and  Target  Attack 
Radar  System 

A 

® 

© 

© 

Source:  GAO  analysis  of  DOD  data. 

Indicates  missing  element 

@  Included  costs  but  no  analysis  of  nonquantifiable  benefits 
(b)  Included  objective  of  document,  not  objective  of  warfighter  or  warfighter  need 
©  Included  either  sensitivity  or  risk  analysis,  but  did  not  include  both 
@  Included  results  but  no  recommendation 

©  Both  the  Navy  Inventory  Control  Point  and  the  Naval  Air  Systems  Command  prepared  a  business  case  analysis 
for  the  F/A-18  E/F  PBL  arrangement.  We  summarize  the  elements  missing  from  both  analyses. 

©  Did  not  analyze  alternative  support  strategies 
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For  three  PBL  arrangements,  the  business  case  analyses  did  not  compare 
alternative  support  options  and  either  evaluated  only  a  single  option  for 
weapon  system  support  or  evaluated  contracting  strategies  instead  of 
alternative  support  arrangements.  For  example,  the  2007  business  case 
analysis  for  the  Joint  Surveillance  and  Target  Attack  Radar  System  did  not 
analyze  the  costs  and  benefits  of  alternative  support  strategies  for  the 
program.  The  business  case  analysis  was  developed  in  response  to  a  2006 
recommendation  from  the  DOD  Office  of  the  Inspector  General  after  a 
review  found  that  the  program  office  had  not  evaluated  alternative  support 
strategies  prior  to  implementing  a  PBL  arrangement  in  2000.23  However, 
the  2007  analysis  covered  only  the  potential  impacts  of  changing  the  type 
of  contract  used  to  obtain  support  for  the  program  from  cost  plus  award 
fee  to  firm  fixed  price.24  The  business  case  analysis  for  the  B-2  also  did  not 
analyze  alternative  support  strategies  but  focused  on  potential  efficiencies 
available  to  the  current  program  through  funding  consolidation,26  funding 
stability,  and  long-term  contracting.  According  to  program  officials,  the 
only  assumption  in  the  analysis  that  actually  occurred  to  some  extent  after 
PBL  implementation  was  funding  consolidation.  Finally,  although  the  C-17 
program  office  developed  a  business  case  analysis  in  2003,  the  DOD  Office 
of  the  Inspector  General  reported  in  2006  that  the  analysis  focused  only  on 
one  support  option  and  did  not  evaluate  the  costs  and  benefits  of  multiple 
support  options.26  In  2007,  a  contract  was  awarded  for  development  of 
another  business  case  analysis  planned  for  completion  prior  to  awarding 
the  next  C-17  support  contract. 


23  Department  of  Defense,  Office  of  the  Inspector  General,  Logistics:  Implementation  of 
Performance-Based  Logistics  for  the  Joint  Surveillance  Target  Attack  Radar  System.,  D- 
2006-105  (Arlington,  Va.,  Aug.  9,  2006). 

"4  The  Federal  Acquisition  Regulation  defines  fixed  price  types  of  contracts  as  providing 
for  a  price  that  is  not  subject  to  any  adjustment  on  the  basis  of  the  contractor’s  cost 
experience  in  performing  the  contract.  FAR  16.202-1.  A  cost-plus-award-fee  contract  is  a 
cost-reimbursement  contract  that  provides  for  a  fee  consisting  of  (1)  a  base  amount  (which 
may  be  zero)  fixed  at  inception  of  the  contract  and  (2)  an  award  amount,  based  upon  a 
judgmental  evaluation  by  the  government,  sufficient  to  provide  motivation  for  excellence  in 
contract  performance.  FAR  16.305. 

25  The  B-2  program  office  received  its  operations  and  maintenance  funding — the  majority  of 
the  funds  used  for  B-2  support — in  separate  elements  of  expense/investment  codes,  which 
had  to  be  used  for  specific  purposes.  In  the  business  case  analysis,  program  officials 
assumed  that  operations  and  maintenance  funds  could  be  used  more  effectively  if  the  funds 
were  consolidated  and  not  separated  into  different  elements  of  expense/investment  codes. 

26  Department  of  Defense,  Office  of  the  Inspector  General,  Acquisition:  Procurement 
Procedures  Used  for  C-l  7  Globemaster  III  Sustainment  Partnership  Total  System. 
Support,  D-2006-101  (Arlington,  Va.,  July  21,  2006). 
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Two  other  important  elements  missing  from  some  analyses  were  an 
evaluation  of  costs  over  the  remaining  life  cycle  of  the  program  and  the 
calculation  of  net  present  value  to  account  for  the  time  value  of  money.27 
For  example,  the  analysis  for  the  Patriot  PBL  arrangement  only  evaluated 
the  costs  and  benefits  over  a  3-year  period.  On  the  other  hand,  while  the 
business  case  analysis  for  the  Assault  Breacher  Vehicle  evaluated  costs 
over  the  20-year  life  cycle  of  the  program,  the  net  present  value  of  the 
alternatives  was  not  calculated.  Four  other  business  case  analyses — those 
prepared  for  the  F/A-18  E/F,  AV-8B  Harrier,  Close-In  Weapon  System,  and 
Consolidated  Automated  Support  System — did  not  include  these  two 
elements  and  several  others,  such  as  sensitivity  or  risk  analysis.  These 
analyses  were  prepared  in  a  similar  format  by  the  Naval  Inventory  Control 
Point,  an  organization  that  provides  supply  support  to  the  Navy,  Marine 
Corps,  and  others.  We  conducted  a  net  present  value  analysis  on  the 
amounts  contained  in  the  Naval  Inventory  Control  Point’s  business  case 
analysis  for  the  F/A-18  E/F  PBL  arrangement  and  found  that  the  PBL 
option  it  chose  was  about  $1.7  million  more  expensive  than  the  alternative 
option.  Its  analysis,  which  did  not  use  net  present  value,  found  that  the 
PBL  option  was  about  $277,000  less  expensive.  The  Naval  Inventory 
Control  Point’s  philosophy  is  that  if  the  costs  of  PBL  are  equal  to  or  less 
than  the  costs  for  government-provided  support,  a  PBL  arrangement  will 
be  used.  Therefore,  if  Naval  Inventory  Control  Point  officials  had 
conducted  a  net  present  value  analysis,  based  on  this  decision  criterion, 
they  would  not  have  implemented  the  PBL  arrangement.  According  to 
Naval  Inventory  Control  Point  officials,  there  is  confusion  in  the  Navy 
regarding  the  purpose  of  the  analyses  they  prepare.  Officials  said  that  the 
analyses  were  conducted  for  internal  decision  making  and  were  not 
intended  to  satisfy  Navy  PBL  policy,  which  places  responsibility  for 
development  of  a  life  cycle  business  case  analysis  on  the  weapon  system 
program  office.28  However,  program  officials  for  the  Close-In  Weapon 
System,  Harrier,  and  Consolidated  Automated  Support  System  did  not 
develop  business  case  analyses  that  evaluated  PBL  implementation  over 
the  remaining  life  cycle  of  the  system. 


2|  According  to  Office  of  Management  and  Budget  Circular  A-94,  net  present  value  is  the 
standard  criterion  for  deciding  whether  a  government  program  can  be  justified  on 
economic  principles. 

28  DOD’s  economic  analysis  instruction  recommends  an  evaluation  of  the  costs  and  benefits 
over  the  life  cycle  of  the  program.  Further,  according  to  the  Department  of  the  Navy 
Performance  Based  Logistics  Guidance  Document  (Jan.  27,  2003),  the  program  office  is 
responsible  for  completion  of  a  business  case  analysis,  and  total  life  cycle  costs  should  be 
used  in  the  analysis. 
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Other  Factors 
Influencing  the 
Quality  of  Business 
Case  Analyses 


Several  other  factors  affected  the  quality  of  the  business  case  analyses  we 
reviewed.  Most  of  the  analyses  we  reviewed  did  not  identify  and  quantify 
the  benefits  that  could  be  expected  from  contractor  incentives  to  increase 
reliability  or  improve  processes  to  reduce  support  costs.  The  only  business 
case  analysis  that  specifically  identified  cost  savings  based  on  contractor 
incentives  to  reduce  costs  was  the  2005  life  cycle  business  case  analysis 
for  the  F/A-18  E/F.  The  life  cycle  analysis  included  an  estimate  of  future 
cost  savings  associated  with  investments  the  contractor  was  willing  to 
make  under  the  arrangement  to  reduce  supply  chain  costs.  In  addition, 
most  of  the  analyses  did  not  recognize  or  quantify  the  costs  associated 
with  the  transfer  of  risk  that  occurs  under  a  performance-based  support 
arrangement.  According  to  the  DOD/Defense  Acquisition  University  PBL 
guide,  PBL  arrangements  transfer  responsibility  for  making  support 
decisions — and  corresponding  risk — to  the  support  provider,  and  risk  is 
one  of  the  major  cost  drivers  for  contractors.  Therefore,  the  use  of 
performance  metrics  could  introduce  a  large  element  of  risk  for  the 
contractor  that  may  be  built  into  the  costs  of  such  an  arrangement.  In 
general,  many  of  the  business  case  analyses  we  reviewed  simply  estimated 
the  costs  for  contract  logistics  support  and  government-provided  support. 
One  exception  was  the  business  case  analysis  for  the  Marine  Corps’ 
Assault  Breacher  Vehicle,  which  attempted  to  address  the  costs  of  risk 
transfer  and  the  benefits  of  contractor  incentive  to  reduce  costs  by 
estimating  the  costs  for  a  traditional  contractor  logistics  support 
arrangement  and  a  performance-based  contractor  logistics  support 
arrangement,  in  addition  to  estimates  for  government-provided  support 
and  performance-based,  government-managed  support. 

Another  business  case  analysis  was  based  on  questionable  assumptions. 
The  2002  business  case  analysis  for  the  Sentinel  program  estimated  that 
the  cost  for  the  government  depot  to  overhaul  the  system  was  50  percent 
of  the  total  cost  of  the  system.  The  business  case  analysis  estimated  that 
for  the  alternative  option — a  partnership  between  the  government  and  the 
contractor  with  the  government  depot  performing  enough  work  to  meet 
the  system’s  core  requirements — the  cost  for  an  overhaul  was  only  25 
percent  of  the  system’s  cost.  The  analysis  assumed  that  under  the 
partnership  option  the  overhaul  cost  less  because,  instead  of  complete 
disassembly  and  parts  replacement,  the  system  would  be  inspected  and 
repaired  only  as  necessary.  However,  according  to  an  official  at  the  Army 
depot  that  would  perform  the  overhaul,  the  depot  also  could  have  used  the 
inspect  and  repair  concept  as  the  basis  for  its  maintenance  work. 
Therefore,  this  assumption  in  the  business  case  analysis  may  have 
overstated  the  costs  for  the  government  depot  to  perform  the  work. 
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Lastly,  the  Naval  Air  Systems  Command’s  2005  life  cycle  business  case 
analysis  for  the  F/A-18  E/F  estimated  that  over  the  28-year  life  cycle  of  the 
program,  PBL  support  costs  were  $76  million  more  expensive  than  costs 
for  government-provided  support.  However,  the  business  case  analysis 
estimated  that  Naval  Inventory  Control  Point  surcharges  would  be 
$325  million  less  for  the  PBL  arrangement.  The  Naval  Inventory  Control 
Point  adds  a  surcharge  to  the  cost  of  goods  sold  to  its  customers 
(including  weapon  system  program  offices)  to  recoup  its  expenses,  but 
according  to  officials,  they  do  not  adjust  their  resources  based  on  PBL 
implementation  and  would  still  need  to  recoup  their  expenses  through 
surcharges  to  other  customers.  Therefore,  while  the  F/A-18  program  office 
may  realize  a  $325  million  benefit  from  the  reduced  surcharge,  the  overall 
costs  to  the  Navy  may  remain  the  same.  Including  this  reduced  amount  in 
the  analysis  is  inconsistent  with  DOD’s  economic  analysis  instruction, 
which  states  that  all  measurable  costs  and  benefits  to  the  federal 
government  should  be  included  in  the  analysis.  In  addition,  in  the  2006 
business  case  analysis  prepared  by  the  Naval  Inventory  Control  Point, 
which  estimated  supply  chain  management  costs  for  both  PBL  and 
government-provided  support  for  a  5-year  period  from  2006  through  2011, 
the  Naval  Inventory  Control  Point’s  costs  were  estimated  to  remain  the 
same  under  either  option.  If  the  Naval  Inventory  Control  Point’s  costs  had 
been  the  same  in  the  life  cycle  business  case  analysis  prepared  by  the 
Naval  Air  Systems  Command  for  the  F/A-18  E/F,  the  PBL  arrangement 
would  be  $401  million  more  expensive  than  government  support  for  the 
28-year  period. 


Business  Case  Analyses 
Were  Often  Not  Updated 
for  Decision  Making  After 
PBL  Implementation 


In  2004,  DOD  guidance  recommended  that  business  case  analyses 
continue  throughout  the  weapons  system’s  life  cycle  and  be  updated  at 
key  decision  points  both  to  validate  the  approach  taken  and  to  support 
future  plans.  The  services,  with  the  exception  of  the  Air  Force,  have  also 
issued  policies  on  conducting  such  updates.  However,  even  when  business 
cases  were  prepared,  we  found  that  program  offices  often  did  not  update 
them  in  accordance  with  DOD’s  guidance  and  service  policies,  nor  did  the 
program  offices  validate  them  against  actual  support  costs  for  decision 
making  after  PBL  implementation.  Neither  DOD  nor  the  services  has 
issued  guidance  that  specifies  what  should  occur  when  updating  or 
validating  a  business  case  analysis. 


Army  policy  states  that  PBL  business  case  analyses  shall  be  updated  prior 
to  the  exercise  of  an  option  period  when  there  are  significant  changes  to 
the  performance  period/terms  of  the  contract  or  evaluation  period,  or 
updated  whenever  there  are  major  programmatic  changes  or  at  least  every 
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5  years.29  Program  offices  for  four  of  the  Army  PBL  arrangements  we 
reviewed  had  not  updated  their  business  case  analyses  as  called  for  by 
Army  policy.  For  example,  the  Tube-launched  Optically-tracked  Wire- 
guided  missile  -  Improved  Target  Acquisition  System  program  office 
developed  a  business  case  analysis  in  1998  before  awarding  the  original 
PBL  contract  and  has  not  updated  or  validated  the  original  analysis. 
Further,  officials  negotiated  a  follow-on  PBL  contract  in  2007  after  the 
terms  of  the  original  contract  were  complete.  Although  program  officials 
for  the  Shadow  Tactical  Unmanned  Aircraft  System  had  planned  to 
complete  an  update  to  their  2002  business  case  analysis  by  the  end  of  2007, 
the  effort  was  delayed  and  the  update  is  expected  to  be  completed  before 
the  end  of  2008.  Although  the  Javelin  program  office  implemented  a  PBL 
arrangement  in  January  2004,  the  business  case  analysis  was  developed  in 
2001,  and  program  officials  do  not  have  plans  to  update  the  analysis. 
Additionally,  although  program  office  officials  for  the  Army’s  Sentinel 
weapon  system  had  updated  their  2002  business  case  analysis,  Army 
policy  calls  for  submission  of  the  business  case  analysis  to  both  the  Office 
of  the  Deputy  Assistant  Secretary  of  the  Army  for  Integrated  Logistics 
Support  and  Army  Materiel  Command  Headquarters  for  review  and 
concurrence  and  then  to  the  Program  Executive  Office  for  approval.  In 
February  2008,  before  the  new  analysis  had  been  reviewed  and  formally 
approved,  another  PBL  contract  was  awarded. 

The  Navy’s  2007  business  case  analysis  guidance  calls  for  updates  every  3 
to  5  years  or  when  significant  programmatic  changes  occur.30  Based  on  this 
policy,  the  T-45  program  office  should  complete  an  update  to  its  business 
case  analysis  for  the  PBL  arrangement  for  support  of  the  aircraft’s  engines 
by  2008.  Although  program  officials  updated  their  2003  analysis  in  2006 
with  actual  support  costs  and  flying  hours,  they  did  not  expand  the 
analysis  to  account  for  the  remaining  life  cycle  of  the  engines.  The  analysis 
projected  costs  only  through  2008,  the  original  contract  period.  Program 
officials  did  not  plan  to  further  update  the  business  case  analysis  or 
prepare  another  one  because  they  believed  that  it  was  not  required. 


29  Assistant  Secretary  of  the  Army  (Acquisition,  Logistics  and  Technology),  Memorandum, 
Performance-Based  Logistics  Business  Case  Analysis  Policy  (Aug.  18,  2005),  and  Army 
Regulation  700-127,  Integrated  Logistics  Support  (Sept.  27,  2007). 

30  Department  of  the  Navy,  Guide  for  Developing  Performance  Based  Logistics  Business 
Case  Analyses,  P07-006  (Nov.  6,  2007).  The  foreword  to  this  guide  states  that  programs 
need  to  update  their  business  case  analysis  data  every  3  to  5  years,  and  the  body  of  the 
document  states  that  updates  should  occur  when  there  are  significant  programmatic  or 
support  changes. 
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Neither  DOD’s  nor  the  services’  policies  clearly  specify  what  should  occur 
when  a  program  office  updates  or  validates  a  business  case  analysis. 
Although  some  programs  are  conducting  another  business  case  analysis, 
as  mentioned  earlier,  program  officials  for  the  T-45  did  not  plan  to  conduct 
another  analysis  because  they  had  updated  their  analysis  with  actual  data. 
Program  officials  for  the  V-22  engine  updated  their  1998  analysis  in  2004. 
The  update  focused  on  assessing  if  several  of  the  ground  rules, 
assumptions,  and  factors  used  in  the  original  study  were  still  valid  and 
providing  a  preliminary  recommendation  on  pursuing  a  follow-on  PBL 
contract,  from  a  cost  standpoint.  However,  the  entire  analysis  was  not 
updated. 


Lack  of  Specific  Criteria 
and  Inadequate  Internal 
Controls  Led  to  Business 
Case  Weaknesses 


Business  case  analyses  were  inconsistently  used  for  PBL  decision  making 
because  DOD  did  not  require  that  the  analyses  be  conducted  and  updated 
or  provide  specific  criteria  to  guide  their  development.  Further,  with  the 
exception  of  the  Army,  the  services’  have  not  established  effective  internal 
controls  to  ensure  that  the  analyses  are  prepared  in  accordance  with 
service  policies  and  guidance.  As  a  result,  DOD  cannot  ensure  that 
decisions  regarding  weapon  system  support  options  are  consistently 
guided  by  comprehensive  and  sound  analytical  tools. 


DOD  Guidance  Has  Not 
Provided  Specific  Criteria  or 
Requirements  for  Conducting 
Sound  Business  Case  Analyses 


DOD  guidance  has  not  provided  specific  criteria  for  conducting  and 
updating  business  case  analyses  for  PBL  decision  making.  Despite  DOD’s 
preexisting  economic  analysis  instruction  recommending  the  analysis  of 
both  quantitative  and  qualitative  factors,  in  2001  DOD  recommended  the 
development  of  a  business  case  analysis  prior  to  implementing  a  PBL 
arrangement  but  provided  little  criteria  for  conducting  such  an  analysis.  In 
2003,  a  Defense  Business  Board  study  recommended  that  DOD  issue 
standard  guidance  for  the  services  to  take  a  more  consistent  approach  to 
PBL  decision  making  and  also  require  the  use  of  business  case  analyses.31 
In  response,  the  January  2004  Under  Secretary  of  Defense  (Acquisition, 
Technology  and  Logistics)  memorandum  was  issued  containing  “guiding 
principles”  for  business  case  analyses.  The  memorandum  stated  that 
business  case  analyses  “will  evaluate  all  services  or  activities  needed  to 
meet  warfighter  performance  requirements  using  ‘best  value’ 
assessments.”  This  memorandum  also  listed  several  quantitative  and 


31  The  Defense  Business  Board  (also  known  as  the  Defense  Practice  Implementation 
Board)  was  established  by  the  Secretary  of  Defense  in  2001  and  consists  of  private-sector 
executives  with  experience  in  business  management  who  provide  independent  advice  and 
recommendations  on  strategies  for  the  implementation  of  best  business  practices. 
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qualitative  factors32  for  consideration  when  developing  a  business  case 
analysis;  however,  it  did  not  indicate  how  these  factors  were  to  be 
evaluated  or  their  relative  importance  in  the  decision-making  process.  The 
memorandum  also  recommended  that  business  case  analyses  be  updated 
or  repeated  to  validate  the  approach  taken  or  to  support  future  plans,  but 
did  not  provide  specific  guidance  as  to  when  such  updates  should  occur. 
According  to  the  memorandum,  a  DOD  PBL  business  case  analysis 
handbook  was  supposed  to  be  forthcoming.  Later  that  year,  the 
DOD/Defense  Acquisition  University  PBL  guide  was  published.  It  had  two 
pages  dedicated  to  the  business  case  analysis  concept — providing 
additional  criteria  and  also  incorporating  the  guiding  business  case 
analysis  principles.  However,  a  handbook  specifically  for  PBL  business 
case  analyses  was  never  issued. 

In  2003,  when  DOD  incorporated  PBL  implementation  into  DOD  Directive 
5000.1,  which  provides  mandatory  policies  for  all  acquisition  programs,  a 
requirement  to  conduct  and  update  a  business  case  analysis  was  not 
included.  Specifically,  the  directive  only  stated  that  acquisition  managers 
shall  use  performance-based  strategies  for  sustaining  products  and 
services  whenever  feasible  and  that  such  PBL  strategies  shall  optimize 
total  system  availability  while  minimizing  cost  and  logistics  footprint.  Also, 
despite  the  Defense  Business  Board’s  recommendation  later  that  same 
year  to  require  the  use  of  business  case  analyses,  DOD  subsequently 
neither  required  program  managers  to  prepare  the  analyses  prior  to  PBL 
implementation  nor  required  them  to  update  the  analyses  after 
implementation.  In  fact,  although  most  of  the  services  have  issued  some 
guidance  and  requirements  for  business  case  analyses,  the  current  Defense 
Acquisition  Guidebook33  no  longer  specifically  refers  to  a  business  case 
analysis,  but  rather  recommends  the  development  of  a  “support  strategy 
analysis”  as  part  of  the  PBL  implementation  process.  According  to  the 
guidebook,  the  support  strategy  analysis  can  be  a  business  case  analysis, 
economic  analysis,  decision-tree  analysis,  or  other  best-value-type 
assessment. 


32  The  factors  listed  are  cost  per  unit  of  output  (such  as  mile,  hour,  etc.),  performance 
measures,  asset  ownership,  size  of  footprint,  reliability  growth,  life  cycle  costs,  diminished 
manufacturing  sources  management,  obsolescence  mitigation,  technology  insertion,  risk 
management,  ability  to  synchronize  with  the  Defense  Transportation  System,  existing 
infrastructure,  and  common  consumable  support. 

33  The  Defense  Acquisition  Guidebook  provides  an  interactive,  online  reference  to  policy 
and  discretionary  best  practice.  The  guidebook  is  available  at  https://akss.dau.mil/dag/ 
(accessed  Oct.  27,  2008). 
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Service  Policies  and  Guidance 
for  Conducting  Business  Case 
Analyses  Were  Slow  to 
Develop,  Inconsistent,  and 
Generally  Not  Enforced 
Because  of  Lack  of  Effective 
Internal  Controls 


Another  reason  for  the  inconsistent  use  of  business  case  analyses  is  that 
the  services’  policies  and  guidance  for  conducting  the  analyses  were  slow 
to  develop  and  were  generally  not  enforced  because  of  a  lack  of  effective 
internal  controls.  Moreover,  we  found  inconsistencies  among  the  services’ 
policies  and  guidance.  In  response  to  DOD’s  recommendation  that 
program  offices  conduct  a  business  case  analysis  prior  to  implementing  a 
PBL  arrangement  for  weapon  system  support,  the  services  issued  their 
own  policies  and  guidance.  The  time  frames  for  these  are  summarized  in 
table  4. 
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Table  4:  Service-Issued  Business  Case  Analysis  Policies  and  Guidance 


Service 

2002 

2003 

2004 

2005 

2006 

2007 

2008 

Army 

Assistant 
Secretary  of  the 
Army 

Memorandum3 

U.S.  Army 
Implementation 
Guide  - 
Performance 
Based  Logistics" 

Assistant 
Secretary  of  the 
Army 

Memorandum" 

Assistant 
Secretary  of  the 
Army 

Memorandum" 

Army 

Regulation 

700-127“ 

Navy 

Navy  PBL 
Guidance 
Document' 

Secretary  of  the 
Navy  Instruction 
5000.2C9 

Navy  Guide 
for 

Developing 
PBL  Business 
Case 
Analyses" 

Air  Force 

Air  Force 
Instruction 

63-107' 

Assistant 
Secretary  of  the 
Air  Force 
Memorandum' 

Air  Force 

Instruction 

65-509k 

Air  Force 

Manual 

65-510' 

Marine 

Corps 

Marine  Corps 
Order  4081. 2m 

Source:  GAO  analysis  of  DOD  data. 

“Assistant  Secretary  of  the  Army  (Acquisition,  Logistics  and  Technology),  Memorandum,  Army 
Implementation  of  Performance  Based  Logistics  (PBL)  (Apr.  1, 2002). 

“Deputy  Assistant  Secretary  of  the  Army,  Integrated  Logistics  Support,  U.S.  Army  Implementation 
Guide  -  Performance  Based  Logistics  (PBL)  (Washington,  D.C.:  May  4,  2004). 

“Assistant  Secretary  of  the  Army  (Acquisition,  Logistics  and  Technology),  Memorandum,  Performance 
Based  Logistics  (PBL)  Business  Case  Analysis  (BCA)  Policy  (Aug.  18,  2005). 

“Assistant  Secretary  of  the  Army  (Acquisition,  Logistics  and  Technology),  Memorandum,  Performance 
Based  Logistics  (PBL)  for  Army  Working  Capital  Fund  (AWCF)  Secondary  Items  (Apr.  25,  2006). 

“Army  Regulation  700-127,  Integrated  Logistics  Support,  September  27,  2007. 

'Assistant  Secretary  of  the  Navy  (Research,  Development  and  Acquisition),  Department  of  the  Navy 
Performance  Based  Logistics  (PBL)  Guidance  Document  (Jan.  27,  2003). 

“Navy  Instruction  5000. 2C,  Implementation  and  Operation  of  the  Defense  Acquisition  System  and  the 
Joint  Capabilities  Integration  and  Development  System,  November  1 9,  2004. 

"Department  of  the  Navy,  Guide  for  Developing  Performance  Based  Logistics  Business  Case 
Analyses,  P07-006  (Nov.  6,  2007). 

Air  Force  Instruction  63-107,  Integrated  Product  Support  Planning  and  Assessment,  November  10, 
2004. 

'Assistant  Secretary  of  the  Air  Force  (Financial  Management  and  Comptroller),  Memorandum,  Interim 
Guidance  for  Business  Case  Analysis  (Mar.  18,  2005). 

kAir  Force  Instruction  65-509,  Business  Case  Analysis,  September  19,  2008. 

Air  Force  Manual  65-510,  Business  Case  Analysis  Procedures,  September  22,  2008. 

■"Marine  Corps  Order  4081 .2,  Marine  Corps  Performance  Based  Logistics  (PBL),  January  5,  2007. 
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Although  DOD  recommended  the  use  of  business  case  analyses  in  2001, 
the  services’  business  case  analysis  policies  and  guidance  have  evolved 
over  time.  In  some  cases,  guidance  was  not  issued  until  years  later.  For 
example,  Marine  Corps  policy  did  not  call  for  PBL  business  case  analyses 
until  2007.  Further,  although  the  Air  Force  included  business  case  analyses 
among  mandatory  procedures  in  2004,  these  procedures  were  not  specific. 
The  Air  Force’s  instruction  states  only  that  “the  program  manager  is 
responsible  for  construction  of  a  business  case  analysis  to  determine  the 
best  strategy  for  meeting  the  PBL  goals.”  Final  Air  Force  guidance  for 
business  case  analyses,  including  PBL  business  case  analyses,  was  not 
issued  until  2008.  As  another  example,  the  Army’s  early  business  case 
analysis  guidance  was  general  in  nature,  and  more  specific  policy 
memorandums  were  issued  in  2005  and  2006.  In  2007,  these  policies  were 
included  in  an  Army  regulation. 

Currently,  service  policies  and  guidance  vary  with  respect  to  which 
programs  should  implement  PBL  arrangements,  which  of  those  programs 
shall  conduct  business  case  analyses,  and  how  often  program  managers 
should  update  their  business  case  analyses.  Until  2007  the  services’ 
policies  and  guidance  varied  significantly.  However,  the  issuance  of  Navy 
guidance  and  Marine  Corps  policy  in  2007  resulted  in  more  consistency. 
Table  5  summarizes  the  services’  business  case  analysis  policies  and 
guidance. 
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Table  5:  Current  Service  PBL  and  Business  Case  Analysis  Policies  and  Guidance 

Issue 

Army 

Navy 

Air  Force 

Marine  Corps 

PBL  implementation 

Shall  be  implemented  on  all 
ACAT  programs  where  it  is 
operationally  and 
economically  feasible 

Preferred  for  all  new 
programs  and  fielded 
ACAT  1  and  II  programs 

Shall  be  implemented 
for  new  ACAT  1  and  II 
programs  and  preferred 
for  new  ACAT  III  and 
fielded  programs  if 
practicable 

Applied  to  all  new 
acquisitions  and 
existing  fielded  ACAT 

1  and  II  programs 

Business  case  analyses 

Shall  be  performed  to 
support  PBL 
implementation 

Mandatory  basis  for 
selecting  a  support 
strategy 

Required  responsibility 
of  program  manager 

Shall  be  developed 
for  all  ACAT  1,  II,  III, 
and  IV(T)  programs 
and  for  all  fielded 

ACAT  1  and  II 
programs 

Business  case  analyses  for 
subsystem-  and 
component-level  PBL 
arrangements 

PBL  may  be  implemented 
on  systems,  subsystems, 
secondary  items, 
components,  assemblies, 
subassemblies,  as  well  as 
processes,  as  validated  by 
a  business  case  analysis 

Serves  as  a  decision 
support  tool  for  legacy 
subsystem-  and 
component-level  PBL 

Not  specified 

Shall  be  conducted  at 
the  system, 
subsystem,  and 
component  levels 

Business  case  analysis 
updates 

Shall  occur  at  least  every 

5  years,  or  whenever  there 
are  major  programmatic 
changes,  or  prior  to 
exercising  contract  options 
when  there  are  significant 
changes  to  the  agreement 

Guide  suggests  updates 
at  least  every  5  years  or 
when  significant 
programmatic  or  support 
changes  occur 

Not  specified 

Shall  be  revalidated 
at  least  every  5  years 

Source:  GAO  analysis  of  DOD  data. 

Note:  Acquisition  categories  (ACAT)  are  generally  based  on  the  acquisition  program’s  location  in  the 
acquisition  process,  dollar  value,  and  Milestone  Decision  Authority  special  interest.  DOD  Instruction 
5000.2,  Operation  of  the  Defense  Acquisition  System,  enc.  2,  May  1 2,  2003. 


With  the  exception  of  the  Army,  the  services  have  not  established  the 
internal  controls,  including  a  review  and  approval  process,  necessary  to 
ensure  that  business  case  analyses  are  conducted  prior  to  PBL 
implementation  and  updated  after  implementation.  For  example,  the 
Navy’s  2003  guidance  assigns  responsibility  for  reviewing  individual 
business  cases  analyses  to  the  system  commands’  cost  departments. 
However,  the  review  only  occurs  when  requested.  Although  a  recently 
issued  Air  Force  instruction  calls  for  a  formal  review  of  all  business  case 
analyses,  including  those  conducted  for  PBL  arrangements,  that  meet 
certain  criteria,  it  is  unclear  how  many  PBL  business  case  analyses  will 
meet  any  of  the  criteria  and  be  subject  to  this  review.  The  2008  Air  Force 
instruction  calls  for  a  review  of  all  business  case  analyses  that  will  be 
(1)  forwarded  outside  of  the  Air  Force;  (2)  forwarded  to  senior  Air  Force 
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officials,  such  as  the  Secretary  of  the  Air  Force;  and  (3)  provided  for 
weapon  systems  that  require  Defense  Acquisition  Board  or  Air  Force 
Acquisition  Board  approval. 

In  contrast,  Army  policy  states  that  program  managers  shall  report 
semiannually  on  the  status  of  PBL  implementation  and  that  business  case 
analyses  for  acquisition  category  I  and  II  programs  should  be  submitted 
for  review  and  verification  to  multiple  offices — including  Army 
headquarters,  the  Army  Materiel  Command,  and  the  Office  of  the  Deputy 
Assistant  Secretary  of  the  Army  (Cost  and  Economics) — and  for  approval 
to  the  Army  Acquisition  Executive.  In  addition,  business  case  analyses  for 
lower-level  programs  should  be  reviewed  and  approved  but  will  not 
verified  by  the  Office  of  the  Deputy  Assistant  Secretary  of  the  Army  (Cost 
and  Economics),  and  the  approval  authority  is  the  program  executive 
officer  or  commander  of  the  related  life  cycle  management  command. 
While  the  Army’s  policy  first  provided  for  these  internal  controls  in  2005, 
Army  officials  said  that  no  programs  have  yet  passed  the  review  and 
approval  process  completely. 


Impact  of  PBL 
Arrangements  on 
Reducing  Weapon 
System  Support  Costs 
Is  Unclear 


The  extent  to  which  PBL  arrangements  are  reducing  costs  for  weapon 
system  support  is  unclear  and  generally  remains  undocumented  even  after 
several  years  of  PBL  implementation.  A  major  difficulty  in  assessing  the 
cost  impact  of  PBL  arrangements  is  the  lack  of  detailed  and  standardized 
cost  data  maintained  by  the  program  offices.  Various  other  factors,  such  as 
the  lack  of  systems  that  are  supported  by  both  PBL  and  non-PBL  support 
arrangements,  the  lack  of  sound  program  baseline  information,  and 
changing  operational  and  materiel  conditions,  also  limited  our  ability  to 
assess  the  impact  of  PBL  implementation  on  support  costs.  While  the 
overall  cost  impact  was  unclear  because  of  a  lack  of  data  and  these  other 
factors,  the  limited  evidence  on  cost  impact  that  was  available  showed 
mixed  results.  We  did  find  some  evidence  that  a  few  PBL  arrangements 
have  reduced  costs.  However,  we  also  found  that  characteristics  of  DOD’s 
PBL  support  arrangements,  such  as  short-term  contracts  and  unstable 
program  requirements  and  funding,  may  limit  their  potential  to  reduce 
costs.  Further,  DOD  has  not  sufficiently  emphasized  the  potential  to 
reduce  costs  as  a  goal  for  PBL  programs.  As  a  result,  DOD  cannot  be 
assured  that  PBL  arrangements  will  reduce  support  costs  and  provide 
cost-effective  support  for  DOD  systems. 
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Program  Managers  Often 
Lacked  Detailed  and 
Standardized  Cost  Data 


In  2004,  a  memorandum  from  the  Under  Secretary  of  Defense  (Acquisition, 
Technology  and  Logistics)  recognized  the  importance  of  cost  data  for 
contract  management  and  future  cost  estimating  and  price  analysis  and 
stated  that  PBL  contracts  shall  include  cost  reporting  requirements.34 
However,  for  the  PBL  arrangements  we  reviewed,  program  offices  often 
did  not  have  detailed  cost  data  that  would  provide  insights  regarding  what 
the  program  office  was  spending  for  various  aspects  of  the  support 
program — such  as  the  cost  of  depot  maintenance  by  subsystem  and  major 
component  or  the  cost  of  engineering  support,  supply  support,  and 
transportation.  When  cost  data  were  available,  the  level  of  detail  and 
format  of  cost  data  varied  considerably.  This  condition  significantly 
affected  our  ability  to  determine  the  impact  of  the  implementation  of  PBL 
on  the  costs  of  supporting  the  systems  in  our  sample,  as  many  factors 
influence  support  costs. 


For  PBL  arrangements  using  fixed-price  contracts  or  fixed-price  contract 
line  items — DOD’s  “ideal”  type  of  PBL  arrangement — we  found  that 
program  offices  generally  did  not  receive  detailed  cost  data  for  the 
program  and  only  knew  the  overall  amounts  paid  for  support.  Only  two 
program  offices  in  our  sample  obtained  contractor  support  cost  data  for 
their  fixed-price  PBL  arrangements,  and  the  format  and  contents  of  the 
reports  were  very  different.  For  example,  the  F/A-18  E/F  program  office 
obtained  Boeing’s  report  on  fiscal  year  2006  costs,  including 
general/administrative  costs  and  profit,  in  a  detailed  reporting  structure 
approved  by  the  Office  of  the  Secretary  of  Defense  (OSD),  Cost  Analysis 
Improvement  Group.  According  to  program  officials,  an  annual  cost 
reporting  requirement  was  included  in  the  2005  fixed-price  PBL  contract 
to  provide  cost  visibility  into  the  program  and  was  added  at  no  additional 
cost  to  the  government.  In  contrast,  the  B-2  program  office  receives  a 
monthly  funds  utilization  report  that  allocates  the  amount  the  Air  Force 
pays  the  contractor  into  seven  high-level  categories,  such  as  planned  depot 
maintenance. 


Although  the  PBL  arrangements  that  used  cost-reimbursable  contracts 
generally  obtained  more  detailed  cost  data  than  those  with  fixed-price 
contracts,  the  format  and  level  of  detail  also  varied.  For  example,  under 
the  1998  PBL  contract,  the  F-117  program  office  did  not  receive  cost  data 


34  Under  Secretary  of  Defense  (Acquisition,  Technology  and  Logistics),  Memorandum, 
Performance  Based  Logistics:  Purchasing  Using  Performance  Based  Criteria  (Aug.  16, 
2004). 
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in  a  format  that  was  detailed  enough  to  report  in  OSD’s  standard  support 
cost  structure.  The  program  office  subsequently  required  more  detailed 
cost  data  reporting  from  Lockheed  Martin  in  the  2006  PBL  contract.  As 
another  example,  the  2003  C-17  PBL  contract  has  both  fixed-price  and 
cost-reimbursable  elements.  According  to  program  officials,  Boeing  did 
not  report  support  cost  data  at  the  level  of  detail  necessary  to  report  in 
OSD’s  support  cost  structure  under  the  contract.  According  to  an  Air 
Force  Cost  Analysis  Agency  official,  a  cost-reporting  requirement  was 
included  in  the  contract’s  option  years  and  more  detailed  cost  reporting 
will  begin  in  fiscal  year  2009. 


Limited  Available  Evidence  Although  cost  data  were  generally  lacking,  the  limited  available  evidence 
On  Cost  Impact  Indicates  on  cost  impact  showed  mixed  results.  Data  we  reviewed  for  the  two 
Mixed  Results  systems  that  were  managed  by  both  PBL  and  non-PBL  arrangements 

indicate  that  the  PBL  arrangements  were  more  costly,  but  based  on  other 
assessments  of  available  data,  there  are  some  indications  that  PBL 
arrangements  can  reduce  costs.  However,  in  seven  out  of  eight  programs 
we  reviewed  where  follow-on,  fixed-price  PBL  contracts  had  been 
negotiated,  expected  cost  reductions  either  did  not  materialize  or  could 
not  be  determined.  Finally,  we  noted  that  officials  reported  performance 
levels  for  some  programs  that  were  significantly  higher  than  required 
under  the  PBL  arrangement,  but  it  is  unknown  whether  the  required  levels 
could  be  achieved  at  a  lower  cost. 


Two  Comparisons  of  PBL 
Arrangements  to  Traditional 
Government  Support  Show 
That  PBL  Arrangements  Cost 
More 


Of  the  29  programs  we  reviewed,  only  the  F100-PW-220  engine  and  the  KC- 
130J/C-130J  airframes  are  maintained  by  both  PBL  arrangements  and 
traditional  government  support  strategies.  We  found  that  the  Air  Force’s 
traditional  support  arrangement  for  the  F100-PW-220  engine  costs  slightly 
less  than  the  Navy’s  PBL  arrangement  for  the  same  engine.  The  Navy  uses 
the  F100-PW-220  engines  in  its  F-16A/B  aircraft  and  sustains  the  engines 
through  a  PBL  contract  with  Pratt  &  Whitney.  The  Air  Force  uses  the  same 
engines  in  its  F-16  and  F-15  aircraft  and  supports  the  engines  at  the 
Oklahoma  City  Air  Logistics  Center.  The  Air  Force  maintains  an  engine 
total  ownership  cost  estimate  that  includes  all  costs  incurred  (depot-level 
repairables,  general  services  division  (expendable  repair  parts),  depot 
programmed  equipment  maintenance,  organizational-level  maintenance, 
intermediate-level  maintenance,  and  continuous  improvement  program). 
To  compare  the  Navy’s  PBL  costs  with  the  Air  Force’s  engine  total 
ownership  costs,  we  removed  the  costs  associated  with  organizational- 
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level  maintenance  from  the  Air  Force’s  costs.  As  shown  in  figure  2, 
converted  to  costs  per  flight  hour,  the  Navy’s  PBL  costs  were  slightly 
higher  than  the  Air  Force’s  costs  in  constant  fiscal  year  2008  dollars.36 


Figure  2:  Comparison  of  Navy  PBL  Costs  with  Air  Force  Government/Traditional 
Costs 
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Source:  GAO  analysis  of  DOD  data. 


Although  the  cost  difference  appears  to  be  decreasing,  the  Navy’s  5-year 
contract  ended  in  2008  and  a  new  PBL  contract  has  not  yet  been 
negotiated.  The  engines  are  currently  being  supported  under  a  6-month 
extension  to  the  original  contract,  and  the  fixed  price  paid  per  engine 
cycle  is  significantly  higher  than  that  paid  during  the  previous  5  years. 
According  to  Navy  officials,  the  decision  to  contract  with  Pratt  &  Whitney 
for  the  support  of  the  Navy’s  engines  was  not  solely  based  on  costs  but 
was  also  based  on  other  factors,  such  as  turnaround  time  for  engine 


35  Constant  dollars  measure  the  value  of  purchased  goods  and  services  at  price  levels  that 
are  the  same  as  those  in  the  base  or  reference  year.  Constant  dollars  do  no  contain  any 
adjustments  for  inflationary  changes  that  have  occurred  or  are  forecast  to  occur  outside 
the  base  year. 
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repairs.  However,  program  officials  could  not  provide  the  data  on  which 
they  based  their  decision. 

Elements  of  the  Air  Force’s  PBL  arrangement  to  support  the  C-130J 
airframe  are  more  expensive  than  the  support  for  the  KC-130J  airframe 
provided  by  the  Navy.  According  to  Navy  officials,  an  analysis  was 
prepared  in  2005  to  compare  costs  for  alternative  repair  arrangements  to 
determine  whether  to  continue  using  the  Navy’s  repair  capability  or  to 
transition  to  contractor-provided  repair  in  2006.  The  Navy’s  analysis 
concluded  that  the  support  provided  by  the  Naval  Surface  Warfare  Center, 
Crane  Division,  would  cost  43  percent  less  than  the  support  provided  by 
the  contractor.  The  analysis  was  based  on  anticipated  2006  flight  hours, 
actual  government  support  costs  from  2005,  and  the  costs  to  exercise  an 
option  for  repair  under  a  preexisting  contract.  Additionally,  we 
independently  compared  overall  costs  for  inventory  management  and 
repair  of  repairable  components36  and  found  that  the  Air  Force’s  PBL  costs 
on  a  per  flight  hour  basis  for  these  elements  were  significantly  higher  than 
the  Navy’s  costs — approximately  131  percent  higher  in  2006  and  164 
percent  higher  in  2007.  However,  according  to  officials,  several  factors 
account  for  some  of  the  difference.  For  example,  the  Air  Force’s  PBL 
arrangement  includes  36  percent  more  consumable  and  repair  parts  than 
the  Navy’s  support  arrangement,  maintenance  of  support  equipment,  and 
support  for  six  locations,  while  the  Navy’s  arrangement  includes  support 
for  only  three  locations. 

A  Few  Indicators  Showed  That 

PBL  Arrangements  Reduced 

Support  Costs  for  Some 

Programs 


Only  a  few  of  programs  we  reviewed  were  able  to  provide  some  indicators 
of  reduced  weapon  system  support  costs  that  could  be  attributed  to  the 
use  of  a  PBL  arrangement.  As  mentioned  earlier,  some  programs  did  not 
have  a  business  case  analysis  demonstrating  how  current  support  costs 
compared  to  other  support  approaches.  Of  the  nine  PBL  arrangements  that 
had  been  implemented  and  had  a  business  case  analysis  that  looked  at 
alternative  support  options,  only  four  could  be  compared  with  PBL 
contract  costs.  Based  on  this  comparison,  three  of  these  four  PBL 
arrangements  indicate  potential  savings  from  PBL  implementation,  while 
the  fourth  is  more  expensive  than  estimated  in  the  business  case  analysis. 
The  remaining  analyses  could  not  be  compared  to  actual  program  costs 


36  While  we  could  not  identify  comparable  costs  for  all  elements  of  support,  we  did  obtain 
repair  costs  for  repairable  components  for  both  the  Air  Force’s  PBL  arrangement  and  the 
Navy’s  government  support  arrangement. 
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after  PBL  implementation  because  of  programmatic  changes  that  occurred 
after  the  analyses  were  conducted. 

The  2005  business  case  analysis  for  the  Army’s  Patriot  estimated  a  3-year 
cost  savings  of  $1.6  million  from  using  a  PBL  arrangement  to  provide  107 
high-demand  parts.37  According  to  a  program  official,  the  contract  is  in  its 
final  year  and  total  obligations  are  expected  to  be  about  $1  million  less 
than  estimated  in  the  business  case  analysis.  Additionally,  two  business 
case  analyses  prepared  by  the  Naval  Inventory  Control  Point  estimated 
that  supply  chain  management  support  costs  were  reduced  by 
implementing  a  PBL  arrangement.  The  business  case  analyses  projected 
cost  savings  of  about  $2.2  million  for  the  5-year  Close-In  Weapon  System 
PBL  arrangement  awarded  in  2006  and  $1.3  million  for  the  5-year  Harrier 
PBL  arrangement  awarded  in  2007.38  Based  on  actual  contract  costs — and 
if  the  contracts  are  not  modified  in  the  future — the  total  savings  for  these 
programs  are  projected  to  be  $5.2  million  and  $5.8  million,  respectively. 
Although  the  F/A-18  E/F  business  case  analysis  estimated  a  5-year  supply 
chain  management  savings  of  approximately  $1.4  million,  the  actual 
contract  cost  is  about  $1.6  million  more  than  the  estimated  amount  in  the 
analysis.  Given  the  difference,  the  PBL  arrangement  has  not  reduced 
support  costs  for  the  program.  As  previously  noted,  two  of  the  PBL 
arrangements  having  evidence  of  reduced  support  costs  are  managed  by 
the  Naval  Inventory  Control  Point.  This  activity  has  used  PBL 
arrangements  since  fiscal  year  2000  and  has  reported  achieving  cost 
reductions  as  a  result,  using  the  Navy  working  capital  fund  to  issue  longer- 
term,  multiyear  contracts  that  can  extend  up  to  5  years  in  length  to 
support  aircraft  or  ship  subsystems  or  components.  According  to  agency 
officials,  these  longer-term  agreements  have  enabled  the  Naval  Inventory 
Control  Point  to  guarantee  the  contractors  a  more  stable  business  base, 
which  provides  contractors  an  incentive  to  make  investments  to  reduce 
costs.  Overall,  as  a  result  of  using  PBL  arrangements,  Naval  Inventory 
Control  Point  officials  estimate  that  they  have  reduced  support  costs  by 
approximately  $26.7  million  and  $63.8  million — or  2.8  and  5.8  percent — in 
fiscal  years  2006  and  2007,  respectively. 


37  The  business  case  analysis  did  not  calculate  net  present  value  to  take  into  account  the 
time  value  of  money.  If  calculated,  the  projected  savings  in  net  present  value  terms  would 
have  been  about  $684,000. 

38  The  business  case  analyses  did  not  calculate  net  present  value  to  take  into  account  the 
time  value  of  money.  If  calculated,  the  projected  savings  in  net  present  value  terms  would 
have  been  about  $2.3  million  and  $4.1  million,  respectively. 
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Although  the  V-22  program  conducted  a  business  case  analysis  in  1998  to 
estimate  alternative  costs  for  supporting  the  engines  and  projected  savings 
of  $249.5  million39  over  the  53-year  life  cycle  of  the  program,  the  analysis 
did  not  take  into  account  the  time  value  of  money  and  calculate  savings 
based  on  net  present  value.  For  this  and  other  reasons,  we  cannot  validate 
that  the  savings  are  reasonable.  In  addition  to  DOD’s  economic  analysis 
instruction,  guidance  from  the  Office  of  Management  and  Budget  also 
states  that  net  present  value  is  the  standard  criterion  for  deciding  whether 
a  government  program  can  be  justified  on  economic  principles.  In  2004, 
another  analysis  was  prepared  for  the  V-22  engine  program  to  determine 

(1)  if  several  assumptions  used  in  the  1998  analysis  were  still  valid  and 

(2)  the  impact  of  any  changes  to  those  assumptions  on  the  cost  savings 
estimate  for  the  PBL  arrangement.  The  later  analysis  concluded  that 
differences  in  three  of  the  original  assumptions  increased  the  projected 
PBL  cost  savings  to  $305.9  million — an  increase  of  $56.4  million.40  The 
updated  savings  again  were  not  calculated  using  net  present  value. 
Moreover,  although  limited  actual  data  were  available,  the  calculations 
generally  made  adjustments  using  assumptions  that  generated  the 
maximum  potential  savings  for  the  PBL  alternative.  For  example,  when 
adjusting  the  1998  analysis  to  account  for  differences  in  the  costs 
experienced  for  excluded  repairs  (repairs  that  were  not  covered  by  the 
PBL  arrangement),  the  total  potential  PBL  cost  savings  were  increased  by 
$48  million  because  the  average  excluded  repair  cost  was  lower  than 
previously  estimated.  However,  even  though  data  showed  that  excluded 
repairs  occurred  at  a  higher  frequency  than  projected  in  the  original 
analysis,  the  later  analysis  did  not  adjust  for  the  higher  frequency  of 
excluded  repairs.  Thus,  the  savings  calculation  is  questionable,  because 
the  analysis  noted  that  the  frequency  of  these  repairs  could  eliminate  all  of 
the  estimated  cost  savings.  Finally,  the  10-year-old  analysis  has  not  been 
completely  updated  to  estimate  costs  based  on  actual  data. 

The  remainder  of  the  analyses  could  not  be  compared  to  current  PBL 
arrangement  costs  because  of  programmatic  changes  that  occurred  after 
the  analyses  were  conducted.  For  example: 

•  According  to  an  Air  Force  C-130J  program  official  involved  in  the 
development  of  the  2004  business  case  analysis,  the  analysis  was 
conducted  while  the  aircraft  was  supported  by  a  commercial  contract; 


39  The  savings  were  calculated  in  constant  fiscal  year  1996  dollars. 

40  The  savings  were  calculated  in  constant  fiscal  year  1996  dollars. 
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PBL  Cost  Reductions  Either 
Did  Not  Materialize  or  Were 
Not  Documented  for  Most 
Follow-on  Contracts 


therefore,  the  program  office  did  not  have  detailed  cost  data  on  which 
to  base  the  estimate.  The  estimate  was  developed,  in  part,  using  cost 
data  from  other  legacy  programs  and  other  assumptions  that  program 
officials  said  did  not  turn  out  to  be  accurate.  Thus,  though  the  business 
case  analysis  helped  program  officials  develop  the  program’s  support 
strategy,  the  cost  estimates  contained  within  are  not  useful  for 
monitoring  current  program  costs. 

•  The  2002  business  case  analysis  for  the  Army’s  Sentinel  PBL 

arrangement  estimated  costs  for  a  26-year  period  beginning  in  2003 
using  a  fleet  size  ranging  from  126  to  198  radars.  According  to  program 
officials,  since  2003  the  fleet  size  has  ranged  from  140  to  143  radars 
and  additional  radars  are  not  anticipated.  Although  a  new  business 
case  analysis  was  prepared,  it  had  not  completed  the  Army’s  review 
and  approval  process  at  the  time  of  our  review. 

Few  of  the  remaining  programs  in  our  sample  could  document  cost 
reductions  attributable  to  the  use  of  a  PBL  arrangement  after  negotiating  a 
follow-on  fixed-price  contract.  The  PBL  concept  envisions  that  support 
providers  are  incentivized  to  improve  reliability  to  ensure  that 
performance  metrics  are  met  and  reduce  their  costs  to  provide  support  to 
increase  profits — especially  under  fixed-price  arrangements.  To  the  extent 
practicable,  we  examined  how  contract  costs  changed  for  eight  programs 
in  our  sample  that  negotiated  follow-on  contracts  or  priced  previously 
unpriced  contract  options  after  completing  fixed-price  PBL  contracts. 
According  to  officials,  a  variety  of  factors  affected  the  support  costs 
negotiated  in  the  PBL  contracts  that  caused  both  costs  increases  and 
decreases.  Only  one  program  had  decreasing  support  costs  that  program 
officials  attributed  to  the  use  of  a  PBL  arrangement.  One  additional 
program  supported  under  a  cost-plus-award-fee  contract  also  reduced 
support  costs  by  changing  the  metrics  included  in  the  contract.  However, 
these  two  programs  did  not  have  updated  business  case  analyses  that 
analyzed  alternative  support  options  over  the  remaining  life  cycle  of  the 
program.  Finally,  only  one  program  office  had  developed  a  methodology 
for  tracking  and  verifying  reliability  improvements  made  under  the  PBL 
arrangement,  although  this  is  necessary  for  quantifying  the  related  cost 
savings. 

Support  costs  for  the  Navy’s  Consolidated  Automated  Support  System 
have  decreased  over  the  8-year  PBL  arrangement  that  began  in  April  2000. 
Program  officials  attribute  the  cost  reductions  the  program  has 
experienced  to  the  implementation  of  a  PBL  arrangement.  Depending  on 
the  level  of  support  chosen,  the  fixed  price  charged  for  the  annual  support 
of  a  test  station  decreased  from  53  to  20  percent  (constant  2008  dollars) 
from  2000  through  2008.  Program  officials  said  that  they  closely  monitored 
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maintenance  data  and  failure  rates  in  order  to  negotiate  lower  fixed  prices 
where  possible.  In  addition,  officials  said  that  they  were  able  to  increase 
the  number  of  repair  and  consumable  parts  covered  under  the 
arrangement  over  the  years.  According  to  officials,  prior  to  the 
implementation  of  the  PBL  strategy  support  costs  for  the  program  were 
even  higher,  but  officials  were  unable  to  locate  the  contracts. 

Although  support  costs  for  a  few  of  the  other  seven  programs  decreased, 
officials  said  that  there  were  a  number  of  other  factors  that  influenced 
costs,  such  as  changes  in  the  scope  of  work  or  planned  usage  of  the 
systems.  For  example,  according  to  Tube-launched  Optically-tracked  Wire- 
guided  missile  -  Improved  Target  Acquisition  System  program  officials,  a 
variety  of  factors  affected  the  costs  negotiated  in  the  2007  contract,  and 
increased  fleet  size  was  one  factor  that  allowed  them  to  negotiate  lower 
rates  per  system.  In  addition,  when  the  first  fixed-price  PBL  arrangement 
was  implemented  in  2001  the  program  was  fairly  new  with  very  few 
systems,  so  the  program  office  did  not  have  an  extensive  amount  of 
historical  program  data  with  which  to  negotiate.  Since  2001,  the  program 
office  has  collected  actual  data  that  it  used  to  negotiate  lower  rates  in  the 
latest  contract.  However,  according  to  program  officials,  the  contractor 
only  recently  started  making  changes  to  the  system  to  improve  reliability. 
These  improvements  were  not  included  in  negotiations  for  the  2007-2011 
contract  but  have  begun  to  improve  failure  rates  and  are  expected  to 
reduce  costs  in  future  contracts. 

Although  the  Army’s  Shadow  Tactical  Unmanned  Aircraft  System  is  not 
supported  by  a  firm-fixed-price  PBL  contract,  program  officials  for  the 
system  said  that  they  were  able  to  reduce  support  costs  by  changing  the 
performance  metrics  used  in  the  PBL  arrangement.  The  maximum 
amounts  authorized  in  the  annual  cost-reimbursable  PBL  contract  for  the 
support  of  this  system  were  reduced  by  28  percent  from  fiscal  years  2006 
through  2007.  According  to  program  officials,  a  program  office  review  of 
PBL  processes  in  early  fiscal  year  2006  concluded  that  while  the  PBL 
arrangement  was  effective  in  terms  of  meeting  the  performance  levels 
specified  in  the  contract,  it  was  not  cost  efficient  and  costs  associated 
with  the  vehicle’s  high  accident  rate  were  an  area  of  particular  concern.  In 
response,  the  program  office  changed  the  performance  metrics  in  the 
contract  to  encourage  the  contractor  to  improve  reliability  and  reduce  the 
accident  rate,  and  also  to  improve  depot  maintenance  efficiency.  As  the 
accident  rate  improved,  the  program  office  was  able  to  negotiate  for  lower 
support  costs  in  the  2007  PBL  contract. 
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Costs  to  Achieve  Higher-Than- 
Required  Performance  Levels 
Are  Unknown 


Finally,  while  the  2005  life  cycle  business  case  analysis  for  the  F/A-18  E/F 
program  office  estimated  that  support  provided  under  a  PBL  arrangement 
would  be  more  expensive  than  government-provided  support,  program 
officials  for  the  Navy’s  F/A-18  E/F  PBL  arrangement  have  developed  a 
process  to  track  and  document  support  cost  reductions  attributed  to 
contractor  investments  to  improve  reliability  and  reduce  costs.  Program 
officials  said  that  both  the  Navy  and  Boeing  have  funded  initiatives  to 
improve  F/A-18E/F  component  reliability,  maintainability,  and 
supportability  as  part  of  the  Supportability  Cost  Reduction  Initiatives 
program.  Under  the  current  fixed-price  PBL  arrangement,  Boeing  has 
invested  approximately  $11.39  million  to  fund  initiatives  that  officials 
currently  estimate  will  generate  cost  reductions  of  approximately 
$279  million  over  the  remaining  life  cycle  of  the  system.  According  to 
program  officials,  Naval  Air  Systems  Command  cost  analysts  have 
validated  baseline  estimates  and  will  annually  track  the  results  of  the 
initiatives  in  order  to  quantify  actual  support  cost  reductions  attributed  to 
the  investments  in  the  future. 

According  to  program  officials,  eight  of  the  PBL  arrangements  within  our 
sample  of  29  systems  generally  achieved  a  level  of  performance  that 
significantly  exceeded  what  is  required  under  the  contracts.  According  to 
the  DOD/Defense  Acquisition  University  PBL  guide,  PBL  arrangements 
should  be  structured  to  meet  the  needs  of  the  warfighter.  Therefore,  if 
actual  performance  exceeds  what  is  called  for  in  the  PBL  arrangement,  it 
also  exceeds  the  level  of  performance  that  is  needed.  According  to 
program  officials,  for  eight  of  the  PBL  arrangements  we  reviewed,  the 
contractors  significantly  exceeded  some  of  the  contractual  performance 
requirements.  For  example: 

•  Since  2002,  Army  officials  said  that  the  average  annual  operational 
readiness  for  the  Tube-launched,  Optically-tracked,  Wire-guided 
missile  -  Improved  Target  Acquisition  System  has  not  been  below  99 
percent,  and  the  system’s  operational  readiness  has  averaged  100 
percent  since  2004.  According  to  a  program  official,  the  Army’s 
readiness  standard  for  this  system  is  90  percent.  Despite  the  Army’s 
standard,  it  continued  to  include  a  performance  incentive  that 
encouraged  higher  levels  of  performance  when  negotiating  a  follow-on 
PBL  contract  in  2007.  The  performance  incentive  includes  payment  of 
an  award  fee  that  encourages  operational  readiness  rates  from  91  to 
100  percent,  with  the  highest  award  fee  paid  for  100  percent  average 
operational  readiness. 

•  According  to  officials,  since  early  2005,  monthly  readiness  rates  for  the 
Army’s  Javelin  have  generally  been  measured  above  98  percent. 
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However,  the  PBL  contract  for  support  of  this  system  only  requires  90 
percent  mission  readiness. 

•  Although  the  contractual  requirement  for  parts  availability  for  the 
Navy’s  V-22  engine  PBL  arrangement  has  been  90  percent  since  1998, 
according  to  program  officials,  actual  parts  availability  has 
consistently  averaged  98  to  100  percent. 

•  Similarly,  with  availability  averaging  98  percent  since  2004,  Air  Force 
program  officials  for  the  LITENING  Advanced  Airborne  Targeting  and 
Navigation  Pod  said  that  the  contractor  has  consistently  exceeded  the 
contract  requirement  for  92  percent  system  availability. 

For  programs  where  performance  significantly  exceeded  contractual 
requirements,  it  is  unclear  how  much  extra  was  paid  to  get  the  added 
performance.  Since  the  government  is  paying  for  this  excess  performance, 
then  the  arrangement,  as  structured,  may  not  provide  the  best  value  to  the 
government,  particularly  since  there  are  other  DOD  programs  that  are  not 
funded  at  levels  that  would  be  required  to  achieve  their  desired  level  of 
readiness. 


Characteristics  of  DOD’s 
PBL  Arrangements  May 
Limit  Their  Potential  to 
Reduce  Support  Costs 


Several  characteristics  of  DOD’s  PBL  arrangements  may  limit  their 
potential  to  reduce  costs.  First,  DOD’s  PBL  contracts  are  limited  to 
relatively  short  time  periods,  while  proponents  of  the  PBL  concept  believe 
that  longer-term  PBL  arrangements  are  necessary  to  encourage  support 
providers  to  make  investments  to  improve  reliability.  Second,  in  DOD — 
where  changing  requirements  and  priorities  can  result  in  fluctuations  in 
the  funding  for  support  of  DOD’s  weapon  systems — creating  a  stable  level 
of  funding  is  challenging.  Third,  many  PBL  arrangements  only  transfer 
responsibility  for  inventory  management  to  the  contractor  and  do  not 
transfer  inventory  ownership,  which  reduces  incentives  for  ensuring  a 
correctly  sized  inventory  level.  Finally,  many  of  DOD’s  PBL  arrangements 
do  not  contain  cost  metrics  or  offer  specific  incentives  to  encourage  cost 
reduction  initiatives. 


Short-term  Contracts  According  to  program  officials,  DOD  support  contracts,  including  PBL 

contracts,  that  are  funded  by  operation  and  maintenance  funds41  are 
generally  limited  to  1  year,  and  working-capital-funded42  contracts  are 


41  Operations  and  maintenance  funds  are  typically  available  for  1  year  as  provided  for  by 
the  annual  appropriations  act.  See,  for  example,  Pub.  L.  No.  110-116,  121  Stat.  1295,  1298 
(Nov.  3,  2007). 

42  Working  capital  funds  are  revolving  funds  and  subject  to  the  authorities  stated  in  section 
2208  of  Title  10  of  the  U.  S.  Code. 
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Unstable  Requirements  and 
Funding 


generally  limited  to  5  years,  with  subsequent  option  years  allowed  up  to  a 
total  of  10  years.  However,  according  to  the  DOD/Defense  Acquisition 
University  PBL  guide,  longer-term  PBL  contracts  are  preferred  because  a 
key  aspect  of  PBL  is  the  provision  of  incentives  for  contractors  to  reduce 
costs  over  time  through  increased  reliability  while  still  making  a  profit. 
Further,  contract  length  should  be  sufficient  to  allow  for  an  adequate 
return  on  any  investments  made  to  improve  reliability. 

Officials  from  several  PBL  arrangements  cited  instances  in  which 
reliability  improvements  were  needed  but  contractors  were  hesitant  to 
make  investments  while  under  annual  support  contracts.  For  example, 
Joint  Primary  Air  Training  System  program  officials  said  that  during  the 
original  10-year  PBL  arrangement  that  began  in  1996,  the  contractor  did 
not  make  any  investments  to  improve  unreliable  components.  Although 
officials  were  expecting  the  fixed-price  performance  contract  to  motivate 
the  contractor  to  invest  in  improvements  to  increase  reliability  and 
maximize  profit,  they  found  that  the  contractor  minimized  its  own  costs 
during  the  contract  period  and  passed  on  the  costs  to  improve  the 
reliability  of  components  with  high  failures  to  the  government  when  the 
contract  was  renegotiated.  Our  prior  work  found  that  the  private  sector 
sometimes  used  PBL  contracts  of  10  to  12  years.43 

Stable  requirements  and  funding,  like  longer-term  contracts,  could  enable 
contractors  to  make  reliability  improvements  and  other  business 
decisions,  such  as  long-term  supplier  arrangements,  that  could  improve 
performance  and  reduce  future  support  costs  because  they  have 
reasonable  assurance  of  future  needs.  For  example,  officials  representing 
one  of  the  PBL  arrangements  we  reviewed  credited  stable  funding  for 
much  of  the  program’s  cost  savings.  The  F-117  program  office  estimated 
that  its  arrangement  would  have  cost  over  $80  million  more  if  the  Air 
Force  had  not  agreed  to  stabilize  the  program’s  support  budget  and 
provide  the  necessary  support  funding  each  year  of  the  contract.  However, 
DOD’s  requirements  and  priorities,  and  related  funding,  for  weapon 
system  support  are  not  always  stable.  For  example,  according  to  Army 
officials,  the  Tactical  Airspace  Integration  System’s  PBL  arrangement  was 
affected  by  a  significant  reduction  of  the  program’s  support  budget.  The 
Army  subsequently  requested  that  the  Defense  Acquisition  University 
study  the  implications  of  funding  on  PBL  arrangements  and  prepare  a  case 
study  based  on  this  example.  In  addition,  for  the  last  several  years  some  of 


43  GAO-04-715. 
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DOD’s  Ownership  of  Inventory 


Availability  of  Cost  Metrics  and 
Effective  Incentives 


the  Army’s  PBL  arrangements  we  reviewed  did  not  receive  all  of  their 
support  funds  at  the  beginning  of  the  fiscal  year  but  rather  in  increments 
throughout  the  year.  Program  officials  for  one  Army  system  said  that  at 
one  point  during  fiscal  year  2005,  they  almost  had  to  discontinue  some  of 
the  support  provided  under  the  contract  because  they  did  not  have 
adequate  support  funds.  Additional  funding  was  eventually  made  available 
after  the  program  office  notified  its  major  command  of  the  situation.  Army 
program  officials  said  that  this  funding  instability  further  exacerbates  the 
impact  of  having  short-term  contracts,  since  all  of  the  funds  are  not 
available  to  the  contractor  to  make  business  arrangements  or  investments 
for  reliability  improvements. 

Many  of  the  PBL  arrangements  we  reviewed  only  transferred 
responsibility  for  inventory  management,  not  ownership,  to  the  contractor. 
An  analysis  by  Sang-Hyun  Kim,  Morris  A.  Cohen,  and  Serguei  Netessine  of 
the  Wharton  School,  University  of  Pennsylvania,  suggests  that  the 
efficiency  of  a  PBL  arrangement  depends  heavily  on  the  asset  ownership 
structure:  with  higher  ownership  responsibility,  the  supplier  is  more  likely 
to  spend  more  to  increase  reliability  and  less  for  maintaining  inventory 
levels.44  According  to  this  study,  under  an  arrangement  in  which  the 
contractor  owns  and  manages  the  inventory,  reliability  improvements  and 
inventory  costs  will  both  be  evaluated  in  terms  of  their  ability  to  meet 
performance  metrics  and  minimize  costs.  If  the  PBL  arrangement  only 
includes  inventory  management,  higher  inventory  levels  may  be  used, 
instead  of  investments  to  improve  reliability,  to  meet  performance 
metrics — particularly  those  that  measure  availability — since  inventory 
holding  costs  are  not  incurred  by  the  contractor.  Consequently,  under 
DOD’s  PBL  arrangements,  contractors  may  choose  to  make  fewer 
reliability  improvements. 

Finally,  many  of  DOD’s  PBL  arrangements  do  not  contain  cost  metrics  or 
offer  specific  incentives  to  encourage  reduced  costs.  According  to  an 
August  2004  memorandum  from  the  Under  Secretary  of  Defense 
(Acquisition,  Technology  and  Logistics)  regarding  performance-based 
criteria,  PBL  should  be  constructed  to  purchase  performance,  which  is 
defined  in  terms  of  operational  availability,  operational  reliability,  cost  per 


44  Sang-Hyun  Kim,  Morris  A.  Cohen,  and  Serguei  Netessine,  “Reliability  or  Inventory? 
Contracting  Strategies  for  After-Sales  Product  Support”  (working  paper  for  the  Wharton 
School,  University  of  Pennsylvania,  Philadelphia,  Pa.,  March  2007) 
http.Y/opimweb. wharton. upenn.  edu/ideas/workingpapers.  cfm ?year=2007  (accessed 
Oct.  21,  2008). 
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unit  usage,  logistics  footprint,  and  logistics  response  time.  The  guidance 
recommended  that  PBL  metrics  be  tailored  to  reflect  the  unique 
circumstance  of  the  arrangement,  but  still  support  desired  outcomes  in 
terms  of  the  five  performance  criteria.  A  subsequent  2005  memorandum 
from  the  Under  Secretary  of  Defense  (Acquisition,  Technology  and 
Logistics)  directed  the  use  of  these  metrics  as  the  standard  set  of  metrics 
for  evaluating  overall  total  life  cycle  systems  management. 

Some  of  the  aviation  PBL  arrangements  we  reviewed  negotiated  their 
support  on  a  cost  per  flight  hour  basis.  For  those  that  did  not,  cost  per 
flight  hour  was  generally  not  included  as  part  of  the  contract  performance 
plan,  with  the  exception  of  the  F/A-18  E/F  PBL  arrangement.  For  example, 
the  C-17  program  office  did  not  negotiate  its  contract  on  a  per  flight  hour 
basis  and  does  not  monitor  cost  per  flight  hour  as  part  of  its  PBL 
arrangement.  None  of  the  nonaviation  PBL  arrangements  we  reviewed 
included  cost  metrics  as  part  of  the  PBL  arrangement.  In  addition,  only 
four  of  the  PBL  arrangements  we  reviewed  contained  incentives  for 
reducing  or  controlling  costs.  For  example,  the  F-117  and  Shadow  Tactical 
Unmanned  Aircraft  System  PBL  arrangements  each  included  a  cost¬ 
sharing  provision  where  the  government  and  the  contractor  would  share 
annual  savings  if  actual  costs  were  below  negotiated  costs.  Further, 
officials  said  that  the  award  plan  for  the  F-22  PBL  arrangement  also  will 
consider  how  actual  costs  compare  to  negotiated  costs  when  calculating 
the  amount  of  award  fee  the  contractor  earns  at  the  end  of  the  year. 


DOD  Did  Not  Emphasize 
the  Potential  for  PBL 
Arrangements  to  Reduce 
Costs  or  Require  the 
Collection  and  Reporting 
of  Detailed  Contractor 
Support  Cost  Data 


Although  PBL  arrangements  were  included  in  a  DOD  pilot  program 
intended  to  demonstrate  the  ability  of  various  initiatives  to  reduce  support 
costs,  DOD  did  not  emphasize  this  goal  in  its  guidance  or  requirements  as 
it  established  the  concept  as  the  department’s  preferred  weapon  system 
support  strategy.  In  general,  improved  performance  was  given  greater 
emphasis,  and  we  found  only  a  few  references  to  cost  reduction  in  DOD’s 
guidance  on  implementing  PBL  arrangements.  With  respect  to 
requirements  for  cost  reporting,  DOD  and  the  services  do  not  require  that 
programs  using  PBL  arrangements,  or  other  contractor  logistics  support 
arrangements,  collect  and  report  detailed  cost  data  in  a  consistent, 
standardized  format. 


Since  2001  DOD’s  guidance  regarding  PBL  has  emphasized  higher  levels  of 
readiness  and  stressed  rapid  implementation.  For  example,  in  2001,  when 
DOD  cited  PBL  as  the  preferred  weapon  system  support  strategy,  PBL  was 
described  as  a  strategy  for  achieving  a  higher  level  of  system  readiness 
through  efficient  management  and  direct  accountability.  In  a  2002  Under 
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Secretary  of  Defense  (Acquisition,  Technology  and  Logistics) 
memorandum,  the  services  were  instructed  to  prepare  PBL 
implementation  plans  that  aggressively  pursue  the  earliest  feasible 
program  implementation  end  dates.45  A  January  2004  Under  Secretary  of 
Defense  (Acquisition,  Technology  and  Logistics)  memorandum  stated  that 
PBL  was  the  department’s  near-term  strategy  to  increase  weapon  system 
readiness  through  integrated  logistics  chains  and  public/private 
partnerships.  The  memorandum  contained  guidance  to  implement  PBL 
where  economically  feasible  and  provided  guiding  principles  for  a  best- 
value  assessment.  The  following  month  a  Deputy  Secretary  of  Defense 
memorandum  again  directed  the  services  to  provide  plans  for  aggressively 
implementing  PBL  arrangements.46 

In  contrast  to  DOD’s  clearly  stated  goal  to  reduce  support  costs  in  the  late 
1990s,  we  found  few  references  to  the  potential  for  PBL  to  reduce  support 
costs  since  2001.  DOD  guidance  generally  only  indirectly  refers  to 
potential  PBL  cost  reductions  to  “compress  the  supply  chain”  and  “reduce 
non-value  added  steps.”  In  May  2003,  DOD  Directive  5000.1,  The  Defense 
Acquisition  System,  was  updated  to  emphasize  that  program  managers 
shall  implement  PBL  strategies  “that  optimize  total  system  availability 
while  minimizing  cost  and  logistics  footprint.”  In  March  2004,  an  Under 
Secretary  of  Defense  (Acquisition,  Technology  and  Logistics) 
memorandum  reiterated  that  PBL  was  the  preferred  strategy  and  provided 
criteria  on  which  to  assess  potential  for  PBL  application.47  One  of  the 
criteria  stated  that  the  cost  per  operational  unit  of  performance  (such  as  a 
flying  hour)  should  be  capable  of  being  reduced  through  PBL 
implementation.  Finally,  in  2005,  the  DOD/Defense  Acquisition  University 
PBL  guide  contained  several  references  to  the  potential  for  PBL  to 
improve  reliability  and  reduce  costs. 

Program  offices  often  lacked  detailed  and  standardized  weapon  system 
support  cost  data  because  DOD  has  not  required  them  to  obtain  and  report 
cost  data  from  the  contractors  that  provide  such  support,  including  those 


45  Under  Secretary  of  Defense  (Acquisition,  Technology  and  Logistics),  Memorandum, 
Performance  Based  Logistics  (Feb.  13,  2002). 

46  Deputy  Secretary  of  Defense,  Memorandum,  Implementation  of  the  Defense  Business 
Practice  Implementation  Board  (DBB)  Recommendation  to  the  Senior  Executive  Council 
(SEC)  on  Continued  Progress  on  Performance  Based  Logistics  (Feb.  4,  2004). 

47  Under  Secretary  of  Defense  (Acquisition,  Technology  and  Logistics),  Memorandum, 
Performance  Based  Logistics  (PBL)  and  Business  Case  Analysis  (BCA)  (Mar.  20,  2004). 
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involved  in  PBL  arrangements.  According  to  the  OSD  Office  of  Program 
Analysis  and  Evaluation,  historical  operating  and  support  costs,  organized 
in  a  standard  format,  are  necessary  for  preparation  of  life  cycle  cost 
estimates  for  new  systems,  budget  formulation,  analysis  of  working  capital 
funds,  development  of  business  case  analyses,  and  future  contract 
negotiations.  Until  2007,  DOD’s  guidance  for  structure  of  support  cost 
estimates,  which  is  also  suggested  as  a  defined  presentation  format  for 
historical  operating  and  support  costs,  included  all  contractor  support — 
labor,  materials,  overhead,  and  other  assets — in  one  category,  while 
government-provided  support  was  reported  in  greater  detail  among 
multiple  categories  and  lower-level  subcategories.48  Therefore,  amounts 
paid  for  contractor  support  were  generally  reported  in  the  aggregate.  In 
October  2007,  DOD  changed  its  guidance  to  include  a  more  detailed 
presentation  of  contractor  support  costs  in  the  various  categories,  similar 
to  the  reporting  of  government  support  costs.49  However,  neither  DOD  nor 
the  services  have  required  program  offices  to  obtain  or  report  contractor 
support  costs,  including  PBL  arrangements  with  contractors,  in  this 
format. 

OSD  and  service  officials  are  beginning  to  recognize  the  need  for  further 
visibility  of  the  costs  of  support  provided  by  contractors.  In  late  2006, 
OSD’s  Office  of  Program  Analysis  and  Evaluation  began  a  study  regarding 
the  collection  of  contractor  support  costs  because  the  department 
acknowledged  that  visibility  into  these  costs  in  DOD’s  systems  was 
generally  limited.  Many  of  the  programs  studied  were  PBL  arrangements 
also  included  in  our  sample.  OSD’s  study  also  found  that  program  offices 
often  did  not  have  detailed  cost  data  and,  if  cost  data  were  provided,  the 
data  often  did  not  conform  to,  or  could  not  be  converted  to,  the  standard 
support  cost  structure.  Based  on  the  study  results,  OSD  is  considering 
requiring  contractors  to  report  their  actual  costs  for  providing  logistics 
support,  including  profit  and  general  and  administrative  expenses,  in 
DOD’s  standard  cost  structure.  However,  the  details  of  the  requirement 
and  which  programs  will  be  subject  to  such  reporting  have  not  been 
finalized. 


48  Department  of  Defense,  Office  of  the  Secretary  of  Defense,  Cost  Analysis  Improvement 
Group,  Operating  and  Support  Cost  Estimating  Guide  (Washington,  D.C.,  May  1992). 

49  Department  of  Defense,  Office  of  the  Secretary  of  Defense,  Cost  Analysis  Improvement 
Group,  Operating  and  Support  Cost  Estimating  Guide  (Washington,  D.C.,  October  2007), 
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Similarly,  Air  Force  officials  have  also  recognized  the  limitations  on 
visibility  into  contractor  support  costs  for  weapon  systems.  The  Air  Force 
is  currently  considering  expanding  visibility  by  requiring  that  all 
contractor-supported  programs  report  actual  obligations  for  contractor 
labor  and  materials  (including  PBL  arrangements)  in  each  of  DOD’s  cost 
structure  categories  for  each  aircraft  mission  design  series.  According  to 
Air  Force  Cost  Analysis  Agency  officials,  this  requirement  is  different  from 
the  one  being  considered  by  OSD  in  that  the  Air  Force  will  have  visibility 
over  the  Air  Force’s  costs  for  contractor  support  but  not  the  contractor’s 
actual  costs. 


Use  and 

Characteristics  of 
Availability  Contracts 
by  the  United 
Kingdom’s  Ministry  of 
Defence 


The  United  Kingdom’s  Ministry  of  Defence  also  uses  performance-based 
arrangements  to  support  its  weapon  systems.  Ministry  of  Defence  officials 
refer  to  this  initiative  as  contracting  for  availability.  Similar  to  DOD,  when 
using  availability  contracts  the  Ministry  of  Defence  pays  industry  for 
aircraft,  engines,  or  components  to  be  available  for  military  operations, 
rather  than  paying  for  specific  repairs,  spares,  and  technical  support. 
According  to  officials,  the  use  of  contracting  for  availability  also  started  as 
an  approach  for  reducing  costs  for  weapon  system  support.  Ministry  of 
Defence  officials  said  that  their  current  contracts  for  availability  generally 
provide  support  for  aviation  systems,  such  as  helicopters  and  combat 
aircraft.  Although  there  are  maritime  availability  contracts,  they  said  that 
most  of  the  ministry’s  maritime  availability  contracts  support  specific 
types  of  equipment  rather  than  entire  ships.  In  general,  the  availability 
contracts  used  by  the  ministry  are  significantly  longer  than  those  used  by 
DOD,  and  the  ministry  uses  an  “open  book  accounting”  arrangement  to 
gain  visibility  into  the  contractors’  costs  to  provide  support.  According  to 
officials,  the  annual  budget  for  the  Defence  Equipment  and  Support 
organization  is  approximately  S13  billion,  including  funds  for  conflict 
operations. 


Ministry  of  Defence 
Budget  Pressures  Drove 
Need  for  Support  Cost 
Reductions 


In  1999,  the  United  Kingdom’s  Defence  Logistics  Operation,  one  of  two50 
entities  that  merged  into  the  current  Defence  Equipment  and  Support 
organization,  established  a  goal  to  reduce  costs  20  percent  by  2005/2006. 
According  to  Ministry  of  Defence  officials,  contracting  for  availability 
began  during  this  period  as  a  way  to  maintain  or  improve  performance 
while  assisting  in  achieving  cost  reductions.  They  believe  that  if  industry  is 


50  The  other  organization  was  the  Defence  Procurement  Agency. 
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paid  for  a  given  level  of  availability,  there  are  incentives  to  reduce  support 
chain  costs  and  make  the  weapon  system  more  reliable  and  processes 
more  efficient.  The  cost  reduction  goal  was  a  key  driver  in  the 
transformation  of  the  maintenance,  repair,  and  overhaul  activity  for 
Harrier  and  Tornado  fast  jet  aircraft.  A  member  of  the  Tornado  Integrated 
Project  Team  stated  that  a  number  of  factors  drove  the  support  strategy 
change  for  the  Tornado  aircraft,  but  the  primary  factor  was  the  need  to 
reduce  costs  to  match  budget  reductions;  the  team  identified  availability 
contracting  as  an  effective  way  to  reduce  costs  and  maintain  performance. 
Officials  also  stated  that  the  support  strategies  for  all  of  the  ministry’s 
helicopters  were  changed  because  of  increased  budget  pressures. 

In  2007,  the  United  Kingdom’s  National  Audit  Office  reported  that  the 
Ministry  of  Defence  has  experienced  significant  reductions  in  the  costs  to 
support  its  fast  jets;  the  Tornado  and  Harrier  costs  have  been  reduced 
from  a  total  of  §711  million  in  2001through  2002  to  §328  million  in  2006 
through  2007,  providing  a  cumulative  saving  of  some  §1.4  billion  over  the 
6-year  period.51  The  National  Audit  Office  reported  that  the  savings  were 
achieved  by  working  with  industry  to  reform  traditional  contracts  into 
availability  contracts.  However,  the  report  also  stated  that  the  ministry  did 
not  have  sufficient  data  to  assess  the  impact  of  changes  in  the  pattern  of 
frontline  operations  and  productivity  increases  from  the  use  of  lean 
techniques  on  total  costs.52  National  Audit  Office  officials  with  whom  we 
met  confirmed  that  while  they  could  validate  overall  cost  reductions,  they 
could  not  attribute  the  entire  savings  solely  to  the  use  of  availability 
contracts.  Other  related  initiatives,  such  as  the  reorganization  and 
reduction  of  locations  for  aircraft  repair  and  upgrade,  the  use  of  lean 
techniques,  and  the  use  of  reliability-centered  maintenance,  also 
contributed  to  the  support  cost  reductions. 


Analytical  Tools  Used  for  Ministry  of  Defence  officials  said  that  they  do  not  require  the  use  of 

Decision  Making  availability  contracts  or  promote  their  use  as  the  preferred  strategy. 

According  to  officials,  the  support  strategy  can  and  should  vary  from 
system  to  system  depending  on  the  circumstances;  in  some  cases,  it  may 
be  appropriate  for  government  activities  to  support  some  systems  in  the 
traditional  manner  and  for  others  to  use  contracting  for  availability.  To 


51  National  Audit  Office,  Transforming  Logistics  Support  for  Fast  Jets. 

"2  Lean  techniques  aim  to  identify  and  eliminate  any  activity  that  does  not  add  value  to  the 
end  user  and  make  the  remaining  activity  flow  in  the  most  efficient  sequence  possible. 
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assist  with  the  decision-making  process,  the  Defence  Equipment  and 
Support  organization  developed  a  “support  options  matrix”  for  use  in 
reviewing  current  and  future  support  arrangements.  Officials  said  that  the 
matrix  was  developed  to  assist  with  analyzing  components  of  support  for 
cost  and  performance  drivers,  illustrating  a  range  of  support  options 
differentiated  by  the  gradual  transfer  of  cost  and  performance  drivers  into 
industry  management  and  presenting  a  clear  rationale  for  each  support 
chain  design  in  terms  of  the  benefit  to  be  derived  from  the  transfer  of 
specific  cost  and  performance  drivers  into  industry  management.  In 
addition  to  the  matrix,  a  contractor  capability  assessment  is  also 
completed  to  determine  the  ability  of  industry  to  assume  greater 
management  responsibility.  Finally,  according  to  officials,  before  they 
enter  into  a  contract  for  availability,  two  additional  analyses  are 
conducted.  The  first  is  an  investment  appraisal,  or  an  “internal  value 
benchmark,”  which  calculates  the  lowest  cost  at  which  the  service  could 
be  provided  by  the  government.  The  second  is  a  business  case  analysis, 
which  discusses  the  different  proposals  and  justifies  the  selection  of  the 
proposed  approach.  Officials  noted  that  the  proposed  approach  does  not 
have  to  be  the  lowest-cost  option,  but  is  usually  the  option  that  offers  the 
best  value  solution  overall. 

In  its  2007  report,  the  National  Audit  Office  indicated  that  internal  value 
benchmarks  were  not  developed  consistently  and  recommended 
development  of  improved  guidance  and  consistent  application  of  a 
common  methodology  for  benchmarks  against  which  to  assess  the  value 
of  proposed  availability  contracts.  National  Audit  Office  officials  said  that 
they  found  variance  in  the  quality  of  these  cost  estimates  and  a  shortage  of 
qualified  people  for  cost  modeling.  They  also  pointed  out  that  as  less  and 
less  support  is  provided  by  the  government,  accurate  cost  modeling  for 
use  when  renegotiating  contracts  will  become  more  important,  and  the 
Ministry  of  Defence  needs  to  maintain  or  improve  visibility  of  support 
costs  for  its  weapon  systems. 


Long-term  Contracts  (19- 
25  Years)  Are  Critical  to 
Success 


Defence  Equipment  and  Support  officials  said  that  they  have  found  the 
long-term  nature  of  availability  contracts  a  key  factor  in  reducing  costs 
and  that  annual  contracts  cannot  achieve  the  same  benefits  as  the  longer- 
term  contracts  do.  According  to  officials,  the  long-term  contracts  for 
Tornado  aircraft  and  helicopter  fleets  reduced  costs  because  the 
contractors  were  able  to  stabilize  their  supply  chain  and  obtain  better 
prices  from  the  supplier  base.  The  Ministry  of  Defence  also  found  that 
industry  preferred  long-term  contracts.  In  a  discussion  of  contracting  for 
availability,  the  “Defence  Industrial  Strategy,”  a  white  paper  dated 
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December  2005, 53  stated  that  companies  are  generally  interested  in  using 
availability  contracts  because  it  provides  the  commercial  firms  with 
greater  returns  over  a  longer  period. 

Ministry  of  Defence  officials  provided  us  with  the  following  examples  of 
their  long-term  availability  contracts: 

•  The  Ministry  of  Defence  has  a  10-year  contract  with  AgustaWestland  to 
support  the  Sea  King  helicopter  until  it  is  projected  to  be  removed 
from  service.  The  Ministry  of  Defence  has  priced  the  contract  for  the 
first  5  years,  and  thereafter  it  will  establish  the  price  in  5-year 
increments. 

•  The  Ministry  of  Defence  has  a  23-year  contract  with  VT  Group  to 
support  two  survey  ships  owned  by  the  ministry.  The  contract  has 
price  renegotiation  points  at  the  7-,  15-,  and  20-year  points. 

•  The  Ministry  of  Defence  has  a  19-year  contract  with  BAE  to  support 
the  fleet  of  Tornado  aircraft.  The  ministry  awarded  the  contract  in 
December  2006  and  priced  it  for  the  first  10  years. 

•  The  Ministry  of  Defence  has  a  25-year  contract  with  AgustaWestland  to 
support  the  Merlin  helicopter  until  it  is  projected  to  be  removed  from 
service.  The  price  for  the  initial  5-year  period  of  the  contract  is  fixed, 
and  the  ministry  is  currently  negotiating  prices  for  the  next  5-year 
period  of  performance  that  begins  in  2011. 


Other  Characteristics  Of  Ministry  of  Defence  officials  said  that  other  factors,  such  as  inventory 
Ministry  of  Defence  ownership,  contract  incentives,  and  cost  visibility,  were  also  important 

Availability  Contracts  when  contracting  for  availability.  Officials  told  us  that  they  preferred  to 

transfer  not  only  management  of  inventory  but  also  inventory  ownership 
under  such  arrangements.  They  noted  that  under  some  of  their  current 
availability  contracts  this  had  not  been  possible  for  a  variety  of  reasons. 
Nonetheless,  in  the  future  they  intend  to  pursue  transfer  of  inventory 
ownership  as  much  as  possible.  Examples  of  Ministry  of  Defence 
availability  contracts  where  officials  said  that  inventory  is  owned  by 
industry,  also  known  as  spares  inclusive,  include  a  contract  for  support  of 
two  survey  ships. 

In  addition,  according  to  ministry  officials,  several  of  the  availability 
contracts — including  those  supporting  the  Sea  King  and  Merlin  helicopters 


53  Secretary  of  State  for  Defence,  “Defence  Industrial  Strategy”  (Defence  White  Paper 
presented  to  Parliament,  December  2005). 
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and  Tornado  fast  jets — had  incentives  referred  to  as  gain  share  or  pain 
share.  In  these  types  of  arrangements,  the  contractor  and  government 
share  cost  savings  or  cost  overruns  in  prenegotiated  proportions. 
According  to  officials,  they  found  that  these  types  of  metrics  are  useful  to 
influence  contractor  cost  control  measures  and  provide  an  incentive  for 
industry  to  develop  changes  and  modifications  that  reduce  support  costs. 
Officials  familiar  with  the  Tornado  fast  jet  availability  contract  explained 
that  their  arrangement  included  gain  sharing  and  pain  sharing  on  both  the 
variable  and  fixed-price  portions  of  the  contract. 

Finally,  officials  explained  that  in  many  of  the  Ministry  of  Defence’s 
availability  contracts,  the  concept  of  open  book  accounting  is  employed. 
Open  book  accounting  is  not  a  defined  term  but  is  more  of  a  general 
expression  describing  a  level  of  access  to  accounting  data  that  would  not 
normally  be  available  under  a  conventional  contract.  In  availability 
contracts,  open  book  accounting  allows  government  program  officials  to 
review  the  accounting  records  of  the  contractor.  This  access  is  not  without 
limits.  Officials  said  that  the  level  of  access  must  be  agreed  to  in  advance 
on  a  case-by-case  basis  and  reflects  the  circumstances  of  the  arrangement 
and  the  need  for  access  to  certain  data  to  monitor  performance  or  benefits 
arising  from  the  arrangement.  For  example,  one  contract  may  only  provide 
for  man-hour  data  because  that  is  all  that  needs  to  be  shared  given  the 
circumstances.  However,  another  contract  may  allow  access  to  direct  cost, 
direct  labor  hours,  and  other  rates  and  factors  that  are  relevant  for  the 
work  involved.  According  to  officials,  the  Ministry  of  Defence  has  an  open 
book  accounting  agreement  with  Agusta Westland  for  the  Merlin  contract 
and  the  government  has  full  visibility  of  the  accounts  pertaining  to  Merlin, 
including  overhead  costs.  The  contract  must  explicitly  address  the  data 
access  arrangements  and  not  rely  on  vague  and  undefined  phrases  that 
could  be  open  to  misinterpretation. 


The  Ministry  of  Defence’s 
Long-term  Availability 
Contracts  May  Affect 
Budget  Flexibility 


According  to  the  2007  National  Audit  Office  report,  long-term  availability 
contracts  may  limit  flexibility  to  respond  to  changes  in  resources.  In  the 
past,  integrated  project  team  leaders  in  the  Ministry  of  Defence  had  some 
ability  to  move  funding  between  resource  lines  to  overcome  short-term 
funding  issues.  However,  this  flexibility  is  diminishing  because  of  the 
transition  to  availability  contracts,  as  larger  portions  of  the  budget  are  pre¬ 
allocated  to  fund  these  contracts.  The  Mine  Warfare  Patrol  and 
Hydrographic  Team  also  raised  concerns  about  loss  of  budget  flexibility. 
This  team  is  responsible  for  providing  support  for  2  hydrographic  ships,  1 
patrol  ship  (HMS  Clyde),  3  River  class  ships,  16  mine  hunters,  and  38 
smaller  ships.  The  budget  for  providing  support  to  these  ships  is 
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approximately  MO  million,  with  £18  million  devoted  to  the  long-term 
availability  contracts  for  the  2  survey  ships,  1  patrol  ship,  and  3  River  class 
patrol  ships.  According  to  Ministry  of  Defence  officials,  these 
arrangements  have  for  the  most  part  been  beneficial.  However,  as  they  are 
structured,  these  programs  do  not  allow  for  any  flexibility.  When  the  Mine 
Warfare  Patrol  and  Hydrographic  Team  recently  had  to  absorb  a  20 
percent  budget  cut,  officials  said  that  the  mine  hunter  ships  bore  the  brunt 
of  the  cut  because  they  had  the  majority  of  the  remaining  support  budget 
not  earmarked  for  an  availability  contract.  The  team  views  the  20  percent 
cut  to  its  budget  to  be,  effectively,  a  40  percent  cut  to  the  mine  hunter  ship 
budget.  Defence  Equipment  and  Support  organization  officials  said  that 
they  are  looking  to  add  more  flexibility  to  future  availability  contracts. 

The  Ministry  of  Defence  has  already  incorporated  some  flexibility  in  a  few 
availability  contracts.  Officials  said  that  the  Tornado  contract  contains 
both  fixed-price  elements  for  management  team,  training,  logistics,  and 
information  systems  and  a  variable  price  element  for  flying  hours.  Given 
this,  the  contract  is  fairly  flexible  and  payment  is  based  on  certain  flying 
hour  availability  bands — with  the  bands  ranging  from  70  to  110  percent 
availability  in  10  percent  increments  that  are  agreed  to  annually.  As 
another  example,  officials  explained  that  the  Merlin  contract  provides 
flexibility  in  that  the  prenegotiated  price  is  linked  to  banded  flying  hours 
with  fixed  and  variable  elements.  Under  traditional  contracting,  they 
estimate  that  only  20  percent  of  the  cost  would  vary  with  flying  hours. 

Also,  within  the  basic  contract  parameters  there  is  a  provision  for  surge 
delivery  for  the  Merlin  helicopter.  Finally,  according  to  officials,  the  Sea 
King  helicopter  support  contract  has  a  similar  flexibility  because  there  are 
a  number  of  flying  hour  bands  and  each  band  has  its  own  price.  In  this 
manner,  the  Ministry  of  Defence  can  increase  or  decrease  flying  hours 
without  renegotiating  the  contract.  Officials  pointed  out  that  one 
drawback  is  that  the  price  charged  per  flying  hour  at  the  lower  bands  is 
higher  because  the  contractor  must  be  able  to  cover  fixed  costs  with  fewer 
flight  hours  to  charge.  However,  they  said  that  the  cost  per  flying  hour  is 
still  far  less  than  it  would  have  been  under  a  more  flexible  traditional 
arrangement. 


Conclusions 


While  the  concept  of  using  PBL  support  arrangements  was  intended  to  be 
a  cost  reduction  strategy  as  well  as  one  that  would  result  in  improved 
performance,  DOD’s  emphasis  has  been  more  focused  on  performance  and 
less  focused  on  cost.  DOD  no  longer  emphasizes  reducing  costs  as  a  goal 
for  PBL  programs,  and  DOD’s  implementation  of  PBL,  in  its  current  form, 
does  not  ensure  that  its  PBL  arrangements  are  cost  effective.  DOD’s 
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emphasis  on  the  implementation  of  PBL  as  the  preferred  weapon  system 
support  strategy  has  deemphasized  the  importance  of  the  development  of 
consistent,  comprehensive,  and  sound  business  case  analyses  to  influence 
decisions  regarding  the  use  of  a  PBL  arrangement.  Although  DOD’s 
guidance  recommends  using  business  case  analyses  to  guide  decisions 
about  using  PBL  arrangements  for  weapon  system  support,  the  DOD 
guidance  does  not  require  these  analyses  and  almost  half  of  the  programs 
we  reviewed  either  did  not  perform  a  business  case  analysis  or  did  not 
retain  documentation  of  their  analysis.  Further,  the  quality  of  the  analyses 
of  those  programs  that  had  performed  a  business  case  analysis  varied 
considerably  since  many  were  missing  elements  of  what  DOD  guidance 
recommends  for  sound  economic  analyses.  Additionally,  most  of  those 
analyses  that  should  have  been  updated  had  not  been.  Thus,  DOD  lacks  a 
sound  approach  for  analyzing  whether  proposed  PBL  arrangements  are 
the  most  cost-effective  strategy  for  supporting  weapon  systems.  Without 
instituting  a  more  consistent,  comprehensive,  and  sound  process  on  which 
to  base  decisions  regarding  the  type  of  arrangement  to  be  used  in 
supporting  DOD  systems,  it  is  unlikely  that  the  department  will  be 
successful  in  achieving  the  sizable  savings  that  were  envisioned  when  the 
PBL  concept  was  adopted. 

Assessing  the  cost-effectiveness  of  PBL  programs  also  requires  the 
availability  of  better  cost  data  at  a  level  of  detail  that  would  support  the 
improved  management  of  ongoing  PBL  programs,  including  awarding 
contract  fees,  assessing  performance  versus  the  cost  to  achieve  it, 
evaluating  historical  costs  to  determine  whether  the  status  quo  should  be 
maintained  over  time,  and  making  support  decisions  about  future  follow- 
on  programs.  Such  data  are  usually  not  available  for  PBL  programs, 
limiting  the  ability  of  program  offices  to  make  program  adjustments  or 
take  restructuring  actions  when  appropriate.  Nonetheless,  a  few  program 
offices  have  acquired  data  at  this  level  and  indicate  that  they  obtained 
them  in  a  cost-effective  manner.  Improved  access  to  detailed  cost  data  is 
another  essential  element  in  improving  the  quality  of  data  available  to 
DOD  decision  makers  regarding  the  cost-effectiveness  of  PBL 
arrangements. 


Recommendations  for 
Executive  Action 


To  ensure  that  PBL  arrangements  are  the  most  cost-effective  option  for 
weapon  system  support,  we  recommend  that  the  Secretary  of  Defense 
direct  the  Under  Secretary  of  Defense  (Acquisition,  Technology  and 
Logistics)  to  take  the  following  five  actions: 
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•  revise  DOD’s  Acquisition  Directive  to  require  development  of  a 
business  case  analysis  to  support  the  decision-making  process 
regarding  weapon  system  support  alternatives,  including  PBL; 

•  revise  PBL  business  case  analysis  guidance  to  more  clearly  define 
what  should  be  included  in  a  business  case  analysis  and  to  establish 
specific  criteria  and  methods  for  evaluating  PBL  support 
arrangements,  including  evaluation  at  the  subsystem  and  component 
levels; 

•  revise  PBL  business  case  analysis  guidance  to  more  clearly  define 
when  business  case  analyses  should  be  updated  during  the  weapon 
system  life  cycle; 

•  require  that  each  service  revise  guidance  to  implement  internal 
controls  to  ensure  that  program  offices  prepare  and  update  business 
case  analyses  that  are  comprehensive  and  sound;  and 

•  require  program  offices  to  collect  and  report  cost  data  for  PBL 
arrangements  in  a  consistent,  standardized  format  with  sufficient 
detail  to  support  traditional  cost  analysis  and  effective  program 
management. 


Agency  Comments 
and  Our  Evaluation 


In  written  comments  to  a  draft  of  this  report  (see  app.  II),  DOD  generally 
concurred  with  our  five  recommendations,  noting  that  the  department  is 
committed  to  evaluating  PBL  strategies  using  business  case  analyses  as 
part  of  the  overall  supportability  assessment  made  during  the  development 
stages  of  weapon  system  acquisition  programs.  Specifically,  the 
department  fully  concurred  with  three  recommendations  and  partially 
concurred  with  two. 


DOD  fully  concurred  with  our  first  recommendation  to  revise  DOD’s 
acquisition  directive  to  require  the  development  of  a  business  case 
analysis  to  support  the  decision-making  process  regarding  weapon  system 
support  alternatives,  including  PBL.  DOD  stated  that  the  department  will 
take  steps  to  address  this  issue  in  the  next  iteration  of  the  DOD  Directive 
5000. 1  and  DOD  Instruction  5000.2  acquisition  regulations.  According  to 
DOD’s  response,  this  new  policy  will  require  that  the  use  of  a  business 
case  analysis  be  mandatory  and  that  this  analysis  serve  as  a  sound  basis 
for  the  selected  supportability  strategy. 

In  response  to  our  second  recommendation  to  revise  PBL  business  case 
analysis  guidance  to  clearly  define  what  should  be  included  in  a  business 
case  analysis  and  to  establish  specific  criteria  and  methods  for  evaluating 
PBL  support  arrangements,  DOD  partially  concurred,  stating  that  it 
established  a  Life  Cycle  Product  Support  Assessment  Team  in  September 
2008  to  study  product  support  policy,  guidance,  past  performance,  and 
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results.  As  part  of  the  study,  existing  business  case  analysis  policy  is  being 
reviewed,  and  the  department  will  evaluate  the  team’s  recommendations 
on  providing  specific  criteria  and  methods  for  evaluating  support 
arrangements  and  determine  how  best  to  incorporate  these 
recommendations  into  mandatory  policy.  The  team’s  initial 
recommendations  are  expected  in  April  2009. 

DOD  fully  concurred  with  our  third  recommendation  to  revise  PBL 
business  case  analysis  guidance  to  more  clearly  define  when,  during  the 
weapon  system  life  cycle,  business  case  analyses  should  be  updated. 
According  to  DOD’s  response,  the  department’s  Life  Cycle  Product 
Support  Assessment  Team  will  evaluate  the  appropriate  timing  of  initial 
business  case  analyses  and  follow-on  updates  to  validate  the  life  cycle 
support  approach  for  weapon  systems,  and  the  team’s  recommendations 
will  be  evaluated  for  inclusion  into  mandatory  policy. 

DOD  fully  concurred  with  our  fourth  recommendation  to  require  that  each 
service  revise  guidance  to  implement  internal  controls  to  ensure  that 
program  offices  prepare  and  update  business  case  analyses  that  are 
comprehensive  and  sound.  As  we  noted  in  our  report,  the  Army  has 
already  implemented  a  PBL  business  case  analysis  review  and  approval 
process.  DOD  stated  that  the  Army’s  internal  controls  will  be  reviewed  by 
the  Life  Cycle  Product  Support  Assessment  Team,  which  will  make 
recommendations  for  expansion  for  DOD-wide  governance  policy  as  part 
of  the  team’s  overall  recommendations  expected  in  April  2009. 

DOD  partially  concurred  with  our  fifth  recommendation  to  require 
program  offices  to  collect  and  report  support  cost  data  for  PBL 
arrangements  in  a  consistent,  standardized  format  with  sufficient  detail  to 
support  traditional  cost  analysis  and  effective  program  management.  DOD 
stated  that  a  provision  for  tailored  cost  reporting  for  major  acquisition 
programs  designed  to  facilitate  future  cost  estimating  and  price  analysis 
has  been  included  in  the  draft  DOD  Instruction  5000.2,  which  is  expected 
to  be  approved  in  the  next  30  days.  Additionally,  the  Life  Cycle  Product 
Support  Assessment  Team  is  reviewing  support  cost  reporting  and  cost 
analysis  as  a  part  of  its  ongoing  study.  According  to  DOD’s  response,  the 
ultimate  goal  is  standardized  support  cost  reporting  for  all  life  cycle 
product  support  efforts,  to  include  support  provided  by  government 
activities. 

While  concurring  with  our  recommendations,  DOD’s  response  noted  that 
the  department  disagrees  with  the  assertion  that  the  goal  of  PBL 
arrangements  is  to  reduce  costs.  Rather,  the  primary  goal  of  PBL 


Page  55 


GAO-09-41  Defense  Logistics 


arrangements  is  to  increase  readiness  and  availability  while  reducing 
overall  sustainment  costs  in  the  long  run.  Our  report  recognized  that  the 
current  DOD  Directive  5000.1  provides  that  PBL  arrangements  shall 
optimize  total  system  availability.  However,  our  report  notes  that  this 
directive  also  provides  that  PBL  arrangements  shall  minimize  costs  and 
the  logistics  footprint.  Moreover,  our  report  stated  that  PBL  emerged  from 
a  1999  DOD  study  to  test  logistics  reengineering  concepts  that  placed 
greater  reliance  on  the  private  sector  for  providing  weapon  system 
support  to  both  reduce  support  costs  and  improve  weapon  system 
performance.  Thus,  reducing  costs  was  a  central  focus  of  the  adoption  of 
PBL  as  DOD’s  preferred  support  strategy.  Based  on  our  analysis  in  this 
report,  we  continue  to  believe  that  the  PBL  support  arrangement  concept 
was  intended  to  be  a  cost  reduction  strategy  as  well  as  a  strategy  that 
would  result  in  improved  performance. 

DOD’s  response  also  noted  that  22  of  the  29  programs  we  reviewed 
produced  business  case  analyses  that  enabled  sound  support  strategy 
determinations.  DOD  further  stated  that  for  28  of  the  29  programs,  the  PBL 
strategies  produced  performance  benefits,  readiness  benefits,  or  both,  and 
15  of  the  programs  reflect  cost-neutral  or  savings  benefits  resulting  from 
the  application  of  the  PBL  strategies.  However,  based  on  our  analysis  in 
this  report,  we  continue  to  believe  that  only  20,  rather  than  22,  of  the 
programs  had  business  case  analyses  that  evaluated  PBL  strategies. 
Further,  as  we  stated  in  our  report,  6  of  these  did  not  retain  some  or  all  of 
the  documentation  and  13  were  missing  elements  of  DOD’s  criteria  for 
economic  analyses.  For  example,  we  found  that  for  one  analysis  the  less 
costly  option  would  have  changed  if  the  department  had  calculated  the  net 
present  value  of  the  two  options  considered.  Additionally,  because  the 
department  did  not  document  all  the  potential  support  options  in  the 
business  case  analyses,  it  is  not  possible  to  determine  if  the  most  cost- 
effective  options  were  chosen.  Thus  we  continue  to  question  the  extent  to 
which  these  analyses  enabled  sound  support  strategy  determination. 
Finally,  while  we  recognize  that  the  PBL  arrangements  may  have  produced 
performance  benefits,  readiness  benefits,  or  both,  deficiencies  in  updated 
business  case  analyses  and  detailed  cost  data  did  not  support  an 
assessment  of  support  costs.  Therefore,  it  is  unclear  how  many  of  the 
programs  may  have  actually  had  cost-neutral  or  savings  benefits  resulting 
from  PBL  strategies.  We  continue  to  believe  that  improvements  in 
collection  and  reporting  of  support  cost  data  and  the  updating  of  business 
case  analyses  are  essential  if  DOD  is  to  determine  the  cost-effectiveness  of 
its  PBL  arrangements. 
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We  are  sending  copies  of  this  report  to  interested  congressional 
committees  and  the  Secretary  of  Defense.  The  report  is  also  available  at  no 
charge  on  the  GAO  Web  site  at  http://www.gao.gov. 

If  you  or  your  staff  have  any  questions  regarding  this  report,  please 
contact  me  at  (202)  512-8365  or  solisw@gao.gov.  Contact  points  for  our 
Offices  of  Congressional  Relations  and  Public  Affairs  may  be  found  on  the 
last  page  of  this  report.  Key  contributors  to  the  report  are  listed  in 
appendix  III. 


William  M.  Solis 

Director,  Defense  Capabilities  and  Management 
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To  evaluate  (1)  the  extent  to  which  the  Department  of  Defense  (DOD) 
used  business  case  analyses  to  guide  decisions  regarding  performance 
based  logistics  (PBL)  arrangements  and  (2)  the  impact  PBL  arrangements 
have  had  on  weapon  system  support  costs,  we  selected  a  nonprobability 
sample  of  29  PBL  arrangements  for  weapon  system  support  initiated  from 
1996  through  2007.  The  29  PBL  arrangements  were  selected  from  lists  of 
weapon  systems  supported  by  PBL  arrangements  provided  by  service 
officials.  With  the  exception  of  the  Navy’s,  we  found  that  the  lists  provided 
by  the  services  either  were  not  current  or  contained  inaccuracies,  and  the 
content  of  the  lists  changed  significantly  during  the  course  of  our  review, 
which  affected  our  sample  selection.  We  chose  system-,  subsystem-,  and 
component-level  PBL  arrangements  from  each  of  the  services  based  on 
length  of  time  since  implementation,  location  of  program  office,  dollar 
value,  and  prior  audit  findings.  The  29  PBL  arrangements  we  selected 
constitute  a  nonprobability  sample  and  the  results  are  not  generalizable  to 
the  population  of  PBL  arrangements. 

To  evaluate  the  extent  to  which  DOD  used  business  case  analyses  to  guide 
decisions  regarding  PBL  arrangements,  we  interviewed  officials  regarding 
DOD  and  service  requirements,  policies,  and  guidance  for  business  case 
analyses  since  2001  and  reviewed  applicable  documents.  We  also  reviewed 
DOD’s  1995  economic  analysis  instruction,  which  states  that  analytical 
studies  that  evaluate  the  cost  and  effectiveness  of  weapon  system  support 
are  considered  to  be  “economic  analyses,”  and  determined  that  the 
guidance  is  consistent  with  Office  of  Management  and  Budget  guidance  for 
benefit-cost  analyses  of  federal  programs.1  We  interviewed  program 
officials  to  discuss  any  business  case  analyses  prepared  to  evaluate  the  29 
PBL  arrangements  before  or  after  PBL  implementation  and  examined  the 
analyses  using  the  criteria  contained  in  DOD’s  economic  analysis 
guidance. 

To  evaluate  the  impact  that  PBL  arrangements  have  had  on  weapon 
system  support  costs,  we  interviewed  program  officials  to  discuss  the 
characteristics  of  the  PBL  arrangements,  including  contract  length, 
contract  type,  scope  of  work,  performance  measures,  performance 
incentives  or  disincentives,  and  cost  data  availability.  In  addition,  we 
asked  program  officials  to  identify  support  cost  reductions  that  occurred 
as  a  result  of  PBL  implementation.  If  a  program  had  renewed  a  fixed-price 


1  Office  of  Management  and  Budget  Circular  A-94,  Guidelines  and  Discount  Rates  for 
Benefit-Cost  Analysis  of  Federal  Programs  (Oct.  29,  1992). 
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PBL  arrangement  or  had  finalized  contract  options  that  were  not  priced, 
we  analyzed  the  contracts  for  trends  in  PBL  support  costs.  We  also 
compared  PBL  contract  costs  to  estimated  PBL  support  costs  in  business 
case  analyses,  where  available,  to  determine  how  closely  the  estimates 
matched  the  actual  PBL  arrangement  costs.  We  also  relied  on  previously 
issued  GAO  reports  on  DOD’s  implementation  of  PBL. 

To  analyze  the  use  of  availability  contracts  for  weapon  system  support  by 
the  United  Kingdom’s  Ministry  of  Defence,  we  interviewed  officials  from 
the  Defence  Equipment  and  Support  organization  regarding  policies  or 
requirements  for  availability  contracts  and  trends  regarding  the  use  of 
these  arrangements.  We  also  interviewed  officials  from  programs 
identified  by  the  Ministry  of  Defence  as  using  availability  contracts  for 
weapon  system  support  to  identify  the  characteristics  of  the  specific 
arrangements  and  the  impact  that  the  use  of  these  contracts  had  on 
support  costs.  In  addition,  we  interviewed  National  Audit  Office  officials 
who  reviewed  the  cost  and  performance  of  two  availability  contracts  for 
support  of  fast  jets.  Finally,  we  reviewed  audit  reports  and  other 
documents  from  the  Ministry  of  Defence  and  National  Audit  Office.  We 
obtained  these  data  for  informational  purposes  only  and  did  not 
independently  verify  the  statements  or  data  provided  by  Ministry  of 
Defence  and  National  Audit  Office  officials. 

Specifically,  in  performing  our  work  we  interviewed  officials  and  obtained 
documents  related  to  PBL  at  the  Office  of  the  Under  Secretary  of  Defense 
(Acquisition,  Technology  and  Logistics),  the  Office  of  the  Assistant 
Secretary  of  the  Navy  (Research,  Development  and  Acquisition),  the  Office 
of  the  Assistant  Secretary  of  the  Air  Force  (Acquisition),  the  Office  of  the 
Assistant  Secretary  of  the  Air  Force  (Installations,  Environment  and 
Logistics),  the  Office  of  the  Assistant  Secretary  of  the  Army  (Acquisition, 
Logistics  and  Technology),  the  Marine  Corps  Headquarters,  the  U.S.  Army 
Materiel  Command,  the  U.S.  Army  Aviation  and  Missile  Command,  the  U.S. 
Army  Communications  and  Electronics  Command,  the  Naval  Sea  Systems 
Command,  the  Naval  Air  Systems  Command,  the  Naval  Center  for  Cost 
Analysis,  the  Air  Force  Cost  Analysis  Agency,  the  Air  Force  Directorate  of 
Economics  and  Business  Management,  the  Air  Force  Materiel  Command, 
the  Air  Force  Aeronautical  Systems  Center,  the  Oklahoma  City  Air 
Logistics  Center,  the  Warner  Robins  Air  Logistics  Center,  the  Ogden  Air 
Logistics  Center,  the  United  Kingdom  Ministry  of  Defence,  and  the  United 
Kingdom  National  Audit  Office.  We  conducted  this  performance  audit 
from  February  2007  through  December  2008  in  accordance  with  generally 
accepted  government  auditing  standards.  Those  standards  require  that  we 
plan  and  perform  the  audit  to  obtain  sufficient,  appropriate  evidence  to 
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provide  a  reasonable  basis  for  our  findings  and  conclusions  based  on  our 
audit  objectives.  We  believe  that  the  evidence  obtained  provides  a 
reasonable  basis  for  our  findings  and  conclusions  based  on  our  audit 
objectives. 
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deputy  under  Secretary  of  defense  for 
Logistics  and  Materiel  Readiness 

3500  DEFENSE  PENTAGON 
WASHINGTON,  DC  20301-3500 


December  3,  2008 


Mr.  William  Solis 

Director,  Defense  Capabilities  and  Management 
U.S.  Government  Accountability  Office 
441  G  Street,  N.W. 

Washington,  DC  20548 

Dear  Mr.  Solis, 

This  is  the  Department  of  Defense  (DoD)  response  to  the  GAO  draft  report, 
GAO-09-4 1,  “DEFENSE  LOGISTICS:  Improved  Analysis  and  Cost  Data  Needed  to 
Evaluate  the  Cost  Effectiveness  of  Performance  Based  Logistics,”  dated  November  4, 
2008  (GAO  Code  350975). 

The  Department  generally  concurs  with  the  recommendations  presented  in  the 
report  on  strengthening  Business  Case  Analyses  (BCA)  guidance  for  evaluating 
supportability  strategics.  Further,  the  Department  will  take  steps  to  update  Acquisition 
Policy  documents  such  as  the  5000  series  of  DoD  regulations  to  include  guidance  to 
Program  Managers  on  basing  supportability  decisions  on  a  BCA.  However,  the 
Department  disagrees  with  the  assertion  that  the  goal  of  performance  based  logistics 
(PBL)  arrangements  is  to  reduce  costs.  Rather,  the  primary  goal  of  PBL  arrangements  is 
to  increase  readiness  and  availability  while  reducing  overall  sustainment  costs  in  the  long 
run. 

The  Department  is  committed  to  evaluating  PBL  strategies  via  BCAs  as  part  of  the 
overall  supportability  assessments  made  during  the  development  stages  of  weapon  system 
acquisition  programs.  Moreover,  the  Department  maintains  that  of  the  29  programs 
examined  by  the  GAO,  22  produced  BCAs  that  enabled  sound  support  strategy 
determinations.  Further,  for  28  of  the  29  programs,  the  Military  Departments  assess  that 
the  PBL  strategies  produced  performance  and/or  readiness  benefits  for  the  operational 
warfighter,  and  1 5  of  the  programs  reflect  cost-neutral  or  savings  benefits  resulting  from 
application  of  the  PBL  strategies.  This  number  should  increase  as  data  becomes  available 
on  the  other  programs. 

Detailed  DoD  comments  on  the  draft  GAO  recommendations  are  provided  in  the 
enclosed.  The  DoD  appreciates  the  opportunity  to  comment  on  the  draft  report. 


Sincere! 


Enclosure: 
As  stated 
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GAO  DRAFT  REPORT  -  DATED  NOVEMBER  4.  2008 
GAO  CODE  350975  /GAO-09-41 

"DEFENSE  LOGISTICS:  Improved  Analysis  and  Cost  Data  Needed  to  Evaluate  the  Cost 
Effectiveness  of  Performance  Based  Logistics" 


DEPARTMENT  OF  DEFENSE  COMMENTS 
TO  THE  RECOMMENDATIONS 


RECOMMENDATION  1:  The  GAO  recommends  that  the  Secretary  of  Defense  direct  the 
Under  Secretary  of  Defense  (Acquisition,  Technology  and  Logistics)  to  revise  DoD’s 
acquisition  directive  to  require  development  of  a  business  case  analysis  to  support  the  decision- 
c  __  making  process  regarding  weapon  system  support  alternatives  including  performance  based 

bee  pg.  bo.  logistics. 

POD  RESPONSE:  Concur.  Although  this  is  already  stated  by  Department  policy  in  the 
Performance  Based  Logistics  Guide  as  mentioned  in  the  draft  report,  the  Department  will  take 
steps  to  include  in  the  next  iteration  of  the  DoDD  5000. 1  and  DoDI  5000.2  acquisition 
regulations  policy  that  will  require  use  of  a  business  case  analysis  be  mandatory  and  that  it 
should  serve  as  a  sound  basis  for  the  selected  supportability  strategy. 

RECOMMENDATION  2:  The  GAO  recommends  that  the  Secretary  of  Defense  direct  the 
Under  Secretary  of  Defense  (Acquisition,  Technology  and  Logistics)  to  revise  Performance 
Based  Logistics  (PBL)  business  case  analysis  guidance  to  more  clearly  define  what  should  be 
included  in  a  business  case  analysis  and  to  establish  specific  criteria  and  methods  for 
evaluating  PBL  support  arrangements,  including  evaluation  at  the  subsystem  and  component 
See  pg.  55.  level. 

POD  RESPONSE:  Partially  concur.  As  stated  in  the  draft  report  the  Department  has  issued 
business  case  analysis  guidance  in  the  Department’s  Performance  Based  Logistics  Guide,  in 
past  Department  memos,  and  in  Chapter  5,  “Life  Cycle  Logistics,”  of  the  Defense  Acquisition 
Guidebook.  However,  all  of  these  documents  are  considered  guidance  in  nature  and  are  not 
mandatory.  In  September  2008,  the  Department  established  a  Life  Cycle  Product  Support 
Assessment  Team  composed  of  DoD,  Service,  Industry,  and  Academia  representatives  to 
review  existing  product  support  policy,  guidance,  past  performance,  and  results.  The  goal  of 
this  study  is  to  provide  recommendations  for  furthering  life  cycle  product  support  within  the 
Department,  and  outline  a  plan  for  increasing  the  effectiveness  of  future  life  cycle  product 
support  strategies  for  weapons  systems.  Existing  business  case  analysis  policy  is  being 
reviewed  by  this  group,  and  the  Department  will  evaluate  its  recommendations  on  providing 
specific  criteria  and  methods  for  evaluating  support  arrangements.  The  Department  expects  an 
initial  draft  version  of  the  team's  recommendation  in  April  2009.  The  Department  will  then 
review  the  recommendations  and  determine  how  best  to  structure  the  team's  business  case 
analysis  recommendations  for  inclusion  into  mandatory  policy. 
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See  pg.  55. 


See  pg.  55. 


See  pg.  55. 


RECOMMENDATION  3:  The  GAO  recommends  that  the  Secretary  of  Defense  direct  the 
Under  Secretary  of  Defense  (Acquisition,  Technology  and  Logistics)  to  revise  Performance 
Based  Logistics  (PBL)  business  case  analysis  guidance  to  more  clearly  define  when  business 
case  analyses  should  be  updated  during  the  weapon  systems'  life  cycle. 

POD  RESPONSE:  Concur.  As  stated  in  our  response  to  Recommendation  2,  a  Product 
Support  Assessment  Team  has  been  chartered  by  the  Department  to  evaluate  the  specifics  of 
business  case  analysis  to  include  the  appropriate  timing  of  initial  and  follow-on  updates  to 
validate  the  life  cycle  sustainment  approach  for  weapons  systems.  The  Department  expects  an 
initial  draft  version  of  the  team’s  recommendation  in  April  2009.  The  Department  will  then 
determine  how  best  to  include  mandatory  policy  for  updating  business  case  analyses. 

RECOMMENDATION  4:  The  GAO  recommends  that  the  Secretary  of  Defense  direct  the 
Under  Secretary  of  Defense  (Acquisition,  Technology  and  Logistics)  to  require  that  each 
Service  revise  guidance  to  implement  internal  controls  to  ensure  that  program  offices  prepare 
and  update  business  case  analyses  that  are  comprehensive  and  sound. 

POD  RESPONSE:  Concur.  As  noted  in  the  report  the  Army  has  already  implemented 
internal  controls,  including  a  review  and  approval  process,  necessary  to  ensure  that  businesses 
case  analyses  are  conducted  prior  to  performance  based  logistics  implementation.  The 
Department  will  ensure  the  Product  Support  Assessment  Team  reviews  the  Army’s  processes 
and  make  recommendations  for  expansion  for  DoD-wide  governance  policy  as  part  of  the 
team’s  overall  recommendations  that  are  expected  in  April  2009. 

RECOMMENDATION  5:  The  GAO  recommends  that  the  Secretary  of  Defense  direct  the 
Under  Secretary  of  Defense  (Acquisition,  Technology  and  Logistics)  to  require  program 
offices  to  collect  and  report  cost  data  for  Performance  Based  Logistics  (PBL)  arrangements  in 
a  consistent,  standardized  format  with  sufficient  detail  to  support  traditional  cost  analysis  and 
effective  program  management 

POD  RESPONSE:  Partially  concur.  As  a  first  step,  the  Department  has  included  in  the  draft 
DoDI  5000.2  expected  for  approval  within  the  next  thirty  days  a  requirement  for  the  provision 
of  tailored  cost  reporting  designed  to  facilitate  future  cost  estimating  and  price  analysis  on 
major  acquisition  programs.  Additionally,  the  Product  Support  Assessment  Team  is  reviewing 
life  cycle  product  support  sustainment  cost  reporting  and  cost  analysis  as  specific  study  areas, 
with  the  ultimate  goal  of  standardized  sustainment  cost  reporting  for  all  life  cycle  product 
support  efforts  to  include  organic  activities.  The  Department  expects  to  see  an  initial  draft 
version  of  the  group’s  recommendation  in  April  2009.  The  Department  will  then  review  the 
recommendations  and  determine  how  best  to  structure  them  for  standardized  reporting. 
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GAO’s  Mission 

The  Government  Accountability  Office,  the  audit,  evaluation,  and 
investigative  arm  of  Congress,  exists  to  support  Congress  in  meeting  its 
constitutional  responsibilities  and  to  help  improve  the  performance  and 
accountability  of  the  federal  government  for  the  American  people.  GAO 
examines  the  use  of  public  funds;  evaluates  federal  programs  and  policies; 
and  provides  analyses,  recommendations,  and  other  assistance  to  help 
Congress  make  informed  oversight,  policy,  and  funding  decisions.  GAO’s 
commitment  to  good  government  is  reflected  in  its  core  values  of 
accountability,  integrity,  and  reliability. 

Obtaining  Copies  of 
GAO  Reports  and 
Testimony 

The  fastest  and  easiest  way  to  obtain  copies  of  GAO  documents  at  no  cost 
is  through  GAO’s  Web  site  (www.gao.gov).  Each  weekday  afternoon,  GAO 
posts  on  its  Web  site  newly  released  reports,  testimony,  and 
correspondence.  To  have  GAO  e-mail  you  a  list  of  newly  posted  products, 
go  to  www.gao.gov  and  select  “E-mail  Updates.” 

Order  by  Phone 

The  price  of  each  GAO  publication  reflects  GAO’s  actual  cost  of 
production  and  distribution  and  depends  on  the  number  of  pages  in  the 
publication  and  whether  the  publication  is  printed  in  color  or  black  and 
white.  Pricing  and  ordering  information  is  posted  on  GAO’s  Web  site, 
http://www.gao.gov/ordering.htm. 

Place  orders  by  calling  (202)  512-6000,  toll  free  (866)  801-7077,  or 

TDD  (202)  512-2537. 

Orders  may  be  paid  for  using  American  Express,  Discover  Card, 

MasterCard,  Visa,  check,  or  money  order.  Call  for  additional  information. 

To  Report  Fraud, 
Waste,  and  Abuse  in 
Federal  Programs 

Contact: 

Web  site:  www.gao.gov/fraudnet/fraudnet.htm 

E-mail:  fraudnet@gao.gov 

Automated  answering  system:  (800)  424-5454  or  (202)  512-7470 

Congressional 

Relations 

Ralph  Dawn,  Managing  Director,  dawnr@gao.gov,  (202)  512-4400 

U.S.  Government  Accountability  Office,  441  G  Street  NW,  Room  7125 
Washington,  DC  20548 

Public  Affairs 

Chuck  Young,  Managing  Director,  youngcl@gao.gov,  (202)  512-4800 

U.S.  Government  Accountability  Office,  441  G  Street  NW,  Room  7149 
Washington,  DC  20548 

